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Teacher evaluation is recognized as being one of the most important aspects of 
program evaluation that ensures quality of instruction and contributes to the professional 
growth and development of teachers. Little is known, however, regarding foreign- 
language teachers' and supervisors' perspectives of the effectiveness of the teacher- 
evaluation strategies and processes they use or experience. This study examined the 
views of foreign-language teachers and supervisors to determine which strategies and 
processes informed teachers' professional development. The study also explored foreign- 
language teachers' and supervisors' views regarding the role of the supervisor in 
conducting teacher evaluations. Bass's theoretical framework of transformational 
leadership was used to investigate the role supervisors play in providing feedback to 
teachers as part of teacher evaluation. 

Participants in this mixed-methods study were foreign-language teachers and 
supervisors in the European and Latin American Language School (UEL) at the Defense 
Language Institute Foreign Language Center in Monterey, California. Data were 
collected via a self-administered, paper-and-pencil survey that contained both open- and 
closed-ended questions. The study's results were analyzed quantitatively and 
qualitatively. Respondents' written comments were taken directly from the survey. 

The results of this study indicated that teachers and supervisors agreed that 
classroom observation is the most popular evaluation strategy, followed by student 
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evaluation of teaching, postobservation conferences, and performance reviews. 
Furthermore, teachers and supervisors concurred that postobservation conferences and 
performance reviews were the most effective teacher-evaluation strategies because they 
promoted the teachers' professional development. Participants agreed that teachers 
receive timely feedback from their supervisors; however, they did not agree on the 
effectiveness of various teacher-evaluation strategies and processes. 

The study concluded that (a) many of the teacher-evaluation strategies and 
processes employed in the UEL teacher-evaluation program promoted teachers' 
professional development, (b) some areas of the teacher-evaluation program were 
identified as needing improvement, and (c) supervisors employ transformational 
behaviors when conducting teacher evaluations. 
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CHAPTER I 
THE RESEARCH PROBLEM 
Statement of the Problem 

Teacher evaluation is an important aspect of program evaluation in educational 
settings and is believed to be one of the most complex issues in education (Beerens, 
2000). Evaluation of teachers is essential for understanding effective teaching and 
learning (Danielson & McGreal, 2000). There is general consensus regarding the various 
purposes of teacher evaluation and the fact that it is critical to teacher growth and teacher 
professionalism (Bernstein, 2004). 

It must be acknowledged, however, that there is controversy and debate 
surrounding the topic of teacher evaluation and how it is implemented (Timperley, 1998). 
Much of the controversy concerning teacher evaluation surfaces in discussions regarding 
which evaluation strategies lead to growth and development, and the impact evaluations 
have on personnel actions (Danielson & McGreal, 2000; Marincovich, 1998; Timperley, 
1998). There does not seem to be agreement on the evaluation strategies that best lead to 
professional development and growth. Teachers and administrators agree that evaluation 
of teaching is important and necessary, but there are questions regarding the effectiveness 
of the various teacher-evaluation strategies and processes. 

Another issue found in the teacher-evaluation literature is that teacher evaluation 
has been characterized as consisting of top-down communication and there are criticisms 
that it has not involved the teacher (Danielson & McGreal, 2000). Teacher-evaluation 
scholars posited that teacher evaluation that influences teacher growth and development 
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should include dialogue regarding teaching, reflective practice, and collaboration 
(Beerens, 2000). 

To effect change and improve educational practice, it is important to examine 
teacher performance and competence (Medley, Coker, & Soar, 1984). It is also important 
to mention that teacher evaluation is viewed as crucial to the ongoing professional 
development of teachers in order to improve teaching (Acheson & Gall, 1997; Attinello, 
Lare, & Waters, 2006; Bernstein, 2004; Danielson & McGreal, 2000; Duke & Stiggins, 
1990; Harris, 1986; McGreal, 1995; Smith, 2008; Wang, 2007). Teacher evaluation is 
important for school administrative personnel, community members, and for teachers 
themselves, as all of these groups are interested in effective teaching (Harris, 1986). 
Schools have responded to the public's requests for accountability in educational 
programs and are, therefore, placing more emphasis on teacher evaluation. Consequently, 
it is imperative that educational programs have well-designed and carefully implemented 
teacher-evaluation programs (Chow, Wong, Yeung, & Mo, 2002). 

This study, therefore, examined the perspectives of foreign-language teachers and 
administrators (i.e., department chairs who serve as supervisors of teachers) on the 
effectiveness of the teacher-evaluation program at the Defense Language Institute 
Foreign Language Center (DLIFLC) located in Monterey, California, and governed by 
the Department of the Army (DA). The Institute provides foreign-language instruction to 
U.S. military and civilian personnel who work in the federal sector. DA Regulation 690- 
400 describes the Army's performance-management program, the Total Army 
Performance Evaluation System (TAPES), however, there is not a DLIFLC policy nor are 
there formal guidelines that govern the DLIFLC teacher-evaluation program. Moreover, 
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the effectiveness of the teacher-evaluation program at the Institute has not been 
investigated nor is it known what the perspectives are of DLIFLC foreign-language 
teachers and supervisors regarding the effectiveness of its teacher-evaluation program. 
Important to teacher evaluation is the role of supervisors, who must provide individual 
consideration and effective feedback as transformational leaders in order to move 
teachers toward professional growth and improved teaching performance. 

Much time and energy is expended on teacher-evaluation processes, yet teachers 
have complained that there is no follow up and it is not productive (Beerens, 2000). 
Given the controversy and debate found in the research literature on teacher-evaluation 
strategies, it is important to consider the perspectives of teachers and administrators 
regarding effective teacher-evaluation processes. Specifically, it is imperative that teacher 
evaluation be investigated at DLIFLC, considering the important and vital mission its 
foreign-language teachers help to carry out as they instruct DLIFLC students, the 
majority of whom are U.S. military personnel. 

Purpose of the Study 

The purpose of this survey study was to examine DLIFLC foreign-language 
teachers' and supervisors' views regarding teacher-evaluation strategies and processes 
that contribute to teachers' professional development for the ultimate purpose of 
improving classroom performance. The study asked DLIFLC foreign-language teachers 
and supervisors about various evaluation strategies used such as classroom observation 
and student evaluation of teaching, and how these various strategies contribute to 
teachers' professional development. The study explored teachers' and supervisors' views 
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regarding the evaluation strategies and the role supervisors play in the conduct of the 
evaluations. 

Background and Need for the Study 

DLIFLC is governed by the DA and the Institute's evaluation system is regulated 
by the DA. The appraisal program is known as TAPES. The appraisal system is described 
in DA Regulation 690-400 (1998). This document provides guidance regarding 
implementation of the evaluation system and stipulates that "Planning for professional 
development is an integral part of the performance management process" (p. 5). 

Foreign-language teachers at DLIFLC are evaluated in both formative and 
summative fashion. All teachers are given job-performance standards, documented in the 
Senior System Civilian Evaluation Report Support Form, 7222-1 (DA, 1998), which their 
supervisors discuss with them at the beginning of the rating period. Performance 
standards are assigned to teachers varying according to the teachers' academic rank. For 
example, a professor is held accountable for more academic work in the area of 
professional development than is a senior instructor. Department chairs, who serve as 
foreign-language teachers' supervisors, are asked to conduct four classroom observations 
during the rating period and to provide feedback to the teachers throughout the year. 
Some of the observations are for formative purposes, meaning that supervisors provide 
opportunities for teachers to reflect on observation data, gain insight into their teaching 
performance, and make adjustments as needed to improve their teaching performance. 
Some observations are also for summative purposes. That is, the data obtained from the 
summative classroom observations are used for the annual evaluation, which is 
documented in the Senior System Civilian Evaluation Report, 7222 (DA, 1998), a 
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summative document that becomes part of the teachers' official personnel file. The 
summative evaluations are used annually for merit-pay allocation. They also are used for 
tenure and rank-advancement considerations. 

Additionally, student evaluations of teaching are reviewed by teachers and 
supervisors, as students' comments on the teachers' instructional performance are another 
source of feedback. Student evaluations of teaching are obtained in the early, middle, and 
later stages of the students' program and are used for both formative and summative 
purposes. This feedback is reviewed by the teachers and their supervisors as part of the 
ongoing process of growth and development in teaching performance. 

Feedback from supervisors' classroom observations and student evaluation of 
teaching are the two primary sources of data used by teachers to improve their teaching 
performance. These two evaluation strategies are also the primary sources of data used by 
supervisors for summative ratings. Teachers may prepare portfolios as part of the 
formative-evaluation process; however, they are not required. Action research is required 
for faculty who hold the ranks of Associate Professor and Professor. 

At the end of the evaluation period, teachers are asked to provide input to their 
supervisors for their annual, summative evaluation. The input generally consists of a list 
of the teachers' accomplishments during the year. The input provided by the teacher 
addresses each of the objectives that are in the teachers' performance standards, and the 
input becomes part of the annual evaluation along with the supervisors' narrative 
evaluation. The input is similar to self-evaluation; however, generally teachers do not 
indicate teaching areas in which they believe they need to improve or grow professionally. 
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The research literature on teacher evaluation proposes that effective teacher 
evaluation should lead to the professional growth and development of teachers (Beerens, 
2000; Danielson & McGreal, 2000; Gordon, 2006). Much time and energy is expended in 
the process of evaluating teachers; however, it is not known which teacher-evaluation 
strategies foster professional development. Some DLIFLC supervisors may discuss with 
their teachers how the evaluation might be used for professional growth; however, there 
is not a formal institutional policy or guideline that encourages the connection between 
evaluation and performance improvement or professional development. 

It is important to determine if the DLIFLC teacher-evaluation system is fostering 
objective and effective evaluations that help identify areas in teachers' performance that 
might be improved. Because DLIFLC offers high-stakes language training to military 
personnel who use their language skills to complete a variety of missions related to needs 
identified by the Department of Defense, it is imperative that DLIFLC foreign-language 
teachers and their supervisors engage in continuous efforts to improve instructional 
approaches to ensure students acquire maximum language proficiency. One source of 
valuable data to promote teacher growth and development is the teacher evaluation 
system. 

It is, therefore, critical that the DLIFLC faculty, foreign-language teachers, and 
their supervisors be encouraged to examine the teacher-evaluation practices that are in 
place. It is also important that teachers and supervisors examine the role the evaluation 
practices play in informing professional development and improving teacher performance 
to ensure that the highest quality foreign-language teaching is being provided to the 
DLIFLC foreign-language learner. It is important and relevant to note Beerens' (2000) 
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claim that faculty growth and development must occur in order to see increases in student 
achievement. 

The literature is replete with scholarly research on teacher evaluation in the K-12 
and higher education settings; however, there has been no research on the teacher 
evaluation programs used at the DLIFLC. There is, therefore, a need for this study in 
order to better inform foreign-language teachers, administrators, faculty-development 
specialists, and others involved in teacher-evaluation processes and how these might 
inform professional development and improve teaching performance at the DLIFLC. 

Theoretical Rationale 

Transformational leadership, developed by Bass (1985), served as the theoretical 
rationale for this study. Educational administrators at the DLIFLC who conduct teacher 
evaluations play a vital role as transformational leaders who guide teachers toward 
professional development and improved performance. Transformational leadership is 
recognized as a leadership model that motivates followers and positively impacts on their 
work performance. Additionally, transformational leaders guide their followers toward 
performance beyond expectations (Bass, 1990; Krishnan, 2001; Seltzer & Bass, 1990). 
Transformational leaders focus on the developmental needs of their followers. Bass' 
original model of transformational leadership consisted of four behaviors that contribute 
to transformational leadership: (a) charisma (idealized influence), (b) inspiration 
(inspirational motivation), (c) intellectual stimulation; and (d) individual consideration. 
Each of the behaviors is briefly described; however, it is the fourth behavior, individual 
consideration, that is most directly related to teacher evaluation. 
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The first behavior or characteristic is referred to as charisma or idealized 
influence. Transformational leaders demonstrate this behavior when they ensure their 
followers are aware of the mission and purposes of the organization. They encourage 
their followers to "look beyond their own self interest for the good of the group" (Bass, 
1990, p. 20). Transformational leaders who are charismatic are respected by their 
followers and followers tend to have a sense of trust and confidence in their leaders. 
Humphreys and Einstein (2003) summarized the charismatic influence of this type of 
leader by stating that this type of leadership "is characterized by providing vision and a 
sense of mission, instilling pride in and among the group and gaining respect and trust" (p. 
86). According to Bass and Riggio (2006), transformational leaders who have idealized 
influence take risks, are consistent, focus on doing the right thing, and demonstrate "high 
standards of ethical and moral conduct" (p. 6). Educational administrators at DLIFLC 
who evaluate teachers must thus create an environment in which teachers feel respected 
and have a sense of trust in their evaluator. 

The second characteristic of transformational leaders is inspiration or inspirational 
motivation. Transformational leaders inspire their followers to go beyond what is 
expected by setting high standards. They communicate the vision and serve as role 
models, motivating followers by communicating in a way that energizes. 
Transformational leaders display enthusiasm and optimism and demonstrate commitment 
to goals (Bass & Riggio, 2006). Inspirational motivation is further characterized as being 
a companion of charisma or idealized influence (Humphreys & Einstein, 2003). Thus, 
DLIFLC evaluators of teachers must communicate positively and enthusiastically to 
inspire their teachers to not only achieve the established standards, but to go beyond them. 
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The third characteristic, intellectual stimulation, is reflected when 
transformational leaders stimulate thinking about the organization's objectives and how 
to achieve them. Problem-finding is encouraged and additional characteristics that are 
seen in this area are "creativity, innovation, calculated risk-taking, and careful 
experimentation" (Bass, 2000, p. 26). Intellectual stimulation is further observed in the 
creative problem-solving approaches that transformational leaders demonstrate. By 
getting followers to look at problems differently, transformational leaders encourage 
followers to be more innovative. 

Transformational leaders who exhibit intellectual stimulation "emphasize 
teaching their followers to search for sensible solutions" (Humphreys & Einstein, 2003, p. 
86). An additional characteristic of intellectual stimulation is that followers are 
encouraged to try new approaches and followers are not criticized if their ideas do not 
yield the expected results (Bass & Riggio, 2006). Evaluators of teachers at the DLIFLC 
must find leadership approaches that encourage teachers to try new instructional 
techniques and to do their own problem-solving regarding classroom issues. Through 
performance feedback discussions, DLIFLC supervisors who evaluate foreign-language 
teachers will be able to stimulate creative problem-solving. 

Whereas, the previously indicated behaviors are relevant to teacher evaluation 
processes, it is the fourth behavior, individualized consideration, that is most directly 
related to the feedback and coaching aspects of teacher evaluation in the professional 
development of teachers. Transformational leaders who demonstrate this behavior give 
their followers individual, personalized support and attention. Bass (2000) emphasized 
that leaders who focus on individualized consideration, "treat each individual follower as 
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having different developmental needs. They give and accept feedback as part of the 
learning process. They provide mentoring and coaching" (p. 26). Transformational 
leaders who practice individualized consideration additionally create new learning 
opportunities for their followers and do so in a supportive environment (Bass, 2008). 

Further, transformational leaders take responsibility for developing their followers 
and develop relationships based on trust (Bass & Avolio, 1993; Gillespie, & Mann, 2004; 
Zagorsek, Domovski, & Skerlavaj, 2009). The research literature confirms that 
transformational leaders are particularly attentive to the individual needs of their 
followers while serving as mentors and coaches (Humphreys & Einstein, 2003; Jung, 
Chow, & Wu, 2003; Strang, 2005). Individualized consideration may be observed when a 
transformational leader is supportive, listens effectively, and delegates tasks in order to 
develop followers (Bass & Riggio, 2006). DLIFLC department chairs who supervise 
foreign-language teachers and who are tasked with evaluating teachers must listen to their 
teachers. These supervisors must tailor their coaching approaches to the individual needs 
of teachers to best help the teachers develop and improve their teaching performance. 

Bass (1990) further posited that employees perform better when they are 
supervised by transformational leaders and that employees of transformational leaders are 
also more satisfied with their performance-appraisal system. Bass, furthermore, claimed 
that transformational leaders value the development of their personnel. In the DLIFLC 
teacher-evaluation process, supervisors must serve as transformational leaders. Those 
individuals who supervise and evaluate the foreign-language teaching faculty implement 
a variety of evaluation strategies, providing performance feedback to their teachers, and 
planning the professional development of their personnel. These DLIFLC supervisors are 
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required to integrate professional development into the overall evaluation process. 
Examples of professional development for teachers include training, mentoring, 
participation in professional organizations, and publishing professional articles (DA 
Regulation, 1998). 

Transformational leadership, with a focus on individualized consideration, can 
help both DLIFLC supervisors and teachers as they explore sources of evaluation 
feedback that may help teachers improve their teaching performance. The Institute is 
currently emphasizing, as one of its goals, that faculty strive to improve learner outcomes 
in order to graduate DLIFLC foreign-language students with higher language proficiency 
than was previously required by the military. Based on the needs of the military, faculty 
and supervisors are asked to examine relevant practices, including teacher-evaluation 
systems, to determine if change is needed to improve these practices. 

Transformational leadership can inform the teacher-evaluation practice which is 
relevant to the teacher-supervisor dynamic as teaching performance is examined. This 
mixed-methods study used the theory of transformational leadership to examine the 
perspectives held by DLIFLC foreign-language supervisors and teachers regarding the 
effectiveness of teacher-evaluation strategies and processes. Specifically, the study 
examined which teacher evaluation strategies contribute to the professional development 
of the faculty. 

Research Questions 

The following research questions were asked in order to understand the foreign- 
language teachers' and supervisors' perspectives: 
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1. What evaluation strategies do DLIFLC supervisors most frequently use to 
assess teacher performance? 

2. What are DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, and to what extent do the perspectives of supervisors and 
teachers differ regarding the evaluation strategies and their effectiveness in 
promoting professional development? 

3. What are DLIFLC teachers' perspectives regarding the main objective of the 
evaluation process? 

4. What are DLIFLC supervisors' perspectives regarding the main objective of 
the evaluation process? 

Limitations 

This study was conducted at DLIFLC, a government-sponsored foreign-language 
institute. The foreign-language teachers and supervisors who were employed at DLIFLC 
may differ from other foreign-language teachers and supervisors. Only teachers and 
supervisors who worked in the Undergraduate European and Latin American Language 
School (UEL) were surveyed. The sample was specific in that only those foreign- 
language teachers who were evaluated the previous year were asked to participate in the 
study. Additionally, only those supervisors who conducted performance evaluations of 
their teachers the previous year were asked to participate in the study. The supervisor 
sample was small (n = 9); therefore, this was a limitation. Further, the sample was limited 
to foreign-language teachers and chairs who volunteered to respond to the surveys. 

The survey was administered at DLIFLC 's UEL, which is the school in which the 
teachers, the supervisors, and the researcher worked. Therefore, the responses provided 
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by the participants could have been impacted by the research setting. For example, the 
researcher facilitated several roundtable discussions with UEL supervisors to review 
various aspects of the school's teacher-evaluation systems. It is possible that these 
discussions might have influenced how the supervisors responded to the survey questions. 

Other limitations of the study are that the study employed a survey instrument. 
Participants self-reported the information on the survey and some chose to omit certain 
questions. Further, the study solely examined foreign-language teachers' and supervisors' 
perspectives of teacher-evaluation strategies and processes. Also, the study was limited to 
use of a convenience sample, foreign-language teachers and supervisors in the UEL, one 
of DLIFLC's undergraduate schools, and is therefore, not generalizable to all foreign- 
language teachers and supervisors at DLIFLC. 

Additionally, a caveat to be acknowledged is that respondents may have 
interpreted the survey questions differently from the way the researcher intended the 
questions. Also, the researcher carefully considered the researcher's role as director of the 
language school in which the study took place and was sensitive to the role of power 
when conducting the study. Another limitation is the fact that the researcher may have 
unintentionally biased the study, due to working at the site where the research was 
conducted. As Creswell (2003) acknowledged, there are weaknesses inherent in all 
statistical procedures and research strategies, including surveys. Every attempt was made 
to rigorously follow accurate data-collection and data-analysis methods; nevertheless, 
caution should be taken when interpreting the data and drawing recommendations and 
conclusions from the data. 
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Significance 

There are abundant research studies on the topic of teacher evaluation in the K-12 
and higher education settings. These studies address various aspects of teacher evaluation 
such as problems found in teacher evaluation and the various evaluation strategies used in 
teacher-evaluation programs (Beerens, 2000; Cardno, 1999; Stronge, Richard, & Catano, 
2008). Researchers in the K-12 and higher education environments claimed that teacher 
evaluation should lead to improved teacher performance as well as professional growth 
and development (Danielson & McGreal, 2000; Marincovich, 1998, 1999; Smith, 2008). 
There is not, however, a body of research on teacher evaluation in the context of 
educational institutions in the federal government. Specifically, there are no studies on 
the subject of teacher evaluation at DLIFLC. Furthermore, there are not many studies that 
specifically examine the perspectives of both teachers and supervisors regarding teacher- 
evaluation strategies they believe promote professional development. 

Therefore, there was a need for research regarding federally employed foreign- 
language teachers' and supervisors' perspectives on the effectiveness of performance 
evaluations. This study, therefore, made an important contribution to the field of teacher 
evaluation. Foreign-language teachers and supervisors at DLIFLC participate in 
formative and summative evaluations and they have important viewpoints regarding the 
effectiveness of the evaluation methods they experience and which strategies inform 
teachers' professional development. At DLIFLC there is currently no DLIFLC policy, 
nor do guidelines exist that may facilitate the process of using formative- and summative- 
evaluation data for the improvement of teaching. Also, DLIFLC has no indicators or 
evidence of the effectiveness of the teacher-evaluation processes. It was, therefore, 
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important to investigate the supervisors' and teachers' perspectives on the evaluation 
processes that were familiar to them. 

Additionally, teacher evaluation is critical to the professional development of 
teachers. During the formative stage of the evaluation process foreign-language 
supervisors employ various methods or strategies to collect data for the purpose of 
providing developmental feedback to the teacher. At the end of an annual rating period, 
supervisors review multiple sources of data, such as student evaluations of teaching and 
classroom observations, to prepare the summative evaluations of their faculty. The annual 
or summative evaluation is referred to as a 360-degree process of data review because 
many data sources are considered. 

This study examined DLIFLC foreign-language teachers' and supervisors' views 
regarding the effectiveness of evaluation strategies and evaluation processes and their 
contribution to teachers' professional development. This study should be of special 
interest to government-related school personnel such as provosts, deans, chairs, faculty- 
development specialists, and teachers or other personnel who engage in teacher- 
evaluation processes. Results of the study should inform teacher-evaluation processes in 
order to contribute to teacher competence and improved student-learning outcomes. 

Furthermore, the information gained from this study may potentially lead to the 
creation of a policy or a set of guidelines that can inform the DLIFLC teacher-evaluation 
system. The guidelines may serve as a point of departure for the development of a 
training program for DLIFLC supervisors to better prepare them for their roles as 
effective evaluators of teaching performance. 
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The findings of this study should assist teachers and administrators, especially 
those who supervise foreign-language teachers, and faculty-development specialists, to 
examine current teacher-evaluation strategies and processes and seek alternative, 
improved teacher-evaluation practices that may support improved teaching. Improved 
teaching may lead to the ultimate goal of students' achieving higher foreign-language 
proficiency results. DLIFLC foreign-language teachers who benefit from an effective 
teacher-evaluation program and who thus help students achieve higher proficiency results 
make an important contribution to the defense of the United States. 

Failure to identify effective teacher-evaluation processes may potentially hinder 
both faculty growth and development in the area of teaching performance and gains in 
student achievement. The result would be ineffective teachers in the DLIFLC foreign- 
language teaching environment who are unable to help their learners achieve the 
demanding foreign-language proficiency results required by the armed services who 
sponsor DLIFLC students. 

Definition of Terms 
The following terms are identified and operationalized for this study: 
Action research: Action research is identified as an assessment strategy that 
promotes inquiry through self-reflection and engages teachers in examining their 
instructional practices (Glanz, 1999; Sagor, 2000; Zepeda, 2006). 

Administrator. For the purposes of this study, administrators are the foreign- 
language department chairs who serve as supervisors and who evaluate the performance 
of their teachers. The term administrator was used interchangeably with supervisor and 
department chair. 
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Department chair. Department chairs at DLIFLC have many responsibilities. One 
of their responsibilities is to supervise foreign-language teachers. The term department 
chair was used interchangeably with supervisor and administrator. 

Defense Language Institute Foreign Language Center (DLIFLC): DLIFLC is a 
military language school established in 1941. The DLIFLC is located on the Presidio of 
Monterey in California and is regarded as one of the finest schools for foreign-language 
instruction in the nation (DLIFLC, 2010). 

Effective evaluation programs. Effective evaluation programs are those that lead 
to the professional growth and development of teachers (Beerens, 2000; Danielson & 
McGreal, 2000). Use of objective evidence, multiple data sources, and data sources that 
vary according to the purpose of the evaluation are considered to be effective evaluation 
processes (Peterson, 2006b). Additionally, conditions for effective teacher evaluation 
include goal-orientation, emphasis on systematic communication, and inclusion of a 
climate for quality evaluation. Also, effective evaluation processes require having a 
technically sound evaluation system; that is, a system based on the Personnel Evaluation 
Standards (Joint Committee on Standards for Educational Evaluation [JCSEE], 2009). 

European and Latin American Language School (UEL): UEL is one of eight 
undergraduate language schools at DLIFLC. The study was conducted in UEL. 

Performance appraisal/performance evaluation. The review and evaluation of an 
employee's achievements. The performance appraisal refers to the annual evaluation 
prepared by supervisors. The terms evaluation and appraisal were used interchangeably. 
(DA, 1998). 
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Performance standards. Standards or objectives contain the evaluative criteria 
used to evaluate teachers. Teaching standards state what teachers should know and be 
able to do in their work as teachers. (Danielson & McGreal, 2000). 

Personnel Evaluation Standards. According to the JCSEE (2009), evaluation 
systems that are technically sound include the four basic standards: (a) propriety (i.e., 
ethical and legal considerations, (b) utility (useful and timely), (c) feasibility (i.e., 
efficient and easy to use), and (d) accuracy (i.e., valid and reliable). 

Professional development. Professional development is an outcome of effective 
teacher evaluation (Beerens, 2000); the process that allows teachers the opportunity to 
expand their knowledge and skills to become more competent teachers. In this process 
teachers expand their awareness and abilities in their roles, their context, and their careers 
(Danielson & McGreal, 2000; Duke & Stiggins, 1990). 

Student evaluation of teaching (SET). Students' comments about, for example, 
teachers' instructional practices, preparation, planning, and knowledge. SET is a teacher- 
evaluation strategy that is considered to be an important source of data that provides 
better feedback than others' ratings when the focus is on student performance (Wilkerson, 
Manatt, Rogers, & Maughan, 2000). SET is also considered to be one of the most widely 
used teacher-evaluation strategies, especially in universities (Seldin, 1999). 

Supervisor. At the DLIFLC, department chairs are first-line supervisors who 
evaluate the foreign-language teaching faculty. DLIFLC supervisors are also 
administrators. The term supervisor was used interchangeably with department chair and 
administrator. 
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TAPES (Total Army Performance Evaluation System). The acronym TAPES is 
used throughout the Army system when referring to Army employees' performance 
evaluations. DLIFLC faculty members are Army employees who participate in TAPES. 
Supervisors conduct formative assessments of teachers by reviewing classroom- 
observation data, student evaluations of teaching, and other data relevant to the 
performance of a foreign-language teacher. At the end of a rating year, supervisors 
provide a summative evaluation of teachers (DA, 1998). 

Teacher assessment. Assessment is described by Braskamp and Ory (1994) as 
being equated with appraisal of ability. These authors define assessment in higher 
education as including "the collection, analysis, interpretation, and use of information 
about programs and people" (p. 13). 

Summary 

Teacher evaluation is considered to be an essential part of teaching and learning; 
however, there is controversy and debate regarding the effectiveness of some teacher- 
evaluation systems. Teacher evaluation is referred to as being a top-down, meaningless 
process that creates tension and anxiety in teachers and supervisors alike (Beerens, 2000). 
Furthermore, although there are numerous evaluation strategies that can be used to 
evaluate teacher performance, such as classroom observation, there is not a general 
consensus in the scholarly literature indicating which evaluation strategies lead to teacher 
growth and development. There is consensus, however, that effective teacher evaluation 
should include multiple sources of data (Seldin, 1999). 

Whereas, the teacher-evaluation system at DLIFLC has a DA regulation that 
governs teacher evaluation at the Institute, there is not a local policy nor do formal 
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guidelines exist to provide explicit instructions on the various aspects of the teacher- 
evaluation program. It will be of benefit to the Institute to learn what the perspectives of 
teachers and administrators are regarding the teacher-evaluation processes in use. 

Chapter 1 identified the research problem and the purpose of the study. In the 
background and need for the study section, the DLIFLC teacher-evaluation system was 
described. The importance of examining the effectiveness of teacher-evaluation strategies 
and processes that are part of the DLIFLC evaluation system was noted. Following the 
above sections, the theoretical rationale, the research questions, and the limitations of the 
study were presented. The significance of the study was outlined and definitions of 
relevant terms were provided. 

The following sections of this study consist of a review of the scholarly literature 
regarding various facets of teacher evaluation such as the purposes of teacher evaluation, 
problems in teacher evaluation, strategies used in teacher evaluation, the role of 
supervision and leadership in teacher evaluation, and models to facilitate teacher 
evaluation and development. The study also contains a methodology section which 
describes the research conducted, the instrumentation, and a discussion of the data 
collection and analysis. The study further includes information regarding the study's 
findings and ends with a discussion of the results, a conclusion, implications, and 
recommendations . 
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CHAPTER II 
REVIEW OF THE LITERATURE 
Introduction 

In this section, the literature related to various aspects of teacher evaluations is 
reviewed. The literature review contains five themes that are introduced in various 
sections. The first theme examines the purposes of teacher evaluation. The problems 
encountered in teacher evaluation are reviewed in the second theme. The third theme 
examines the methods or strategies used in teacher evaluation. The fourth theme 
examines the role of supervision and leadership in teacher evaluation, guiding teachers to 
make connections from evaluation to professional growth and development. The final 
theme describes two leadership models that facilitate teacher evaluation and development. 

The scholarly literature on the topic of teacher evaluation that is used in this study 
comes from the K-12 and higher education contexts. The reason for this is that although 
DLIFLC is a U.S. government foreign-language school, when reviewing the teacher 
evaluation strategies used in K-12 and at the university levels, it became evident that 
DLIFLC teacher-evaluation strategies were similar, and in some cases identical to those 
used in these academic settings. Moreover, the goals and objectives of teacher evaluation 
in the public educational environment are the same as those of the DA performance- 
management program; that is, to help teachers grow and develop professionally and to 
improve performance. 

Furthermore, it is important to note that although there is a wealth of literature on 
the topic of teacher evaluation, there is little scholarly literature that examines the 
perspectives of both teachers and supervisors on teacher evaluation. Also, there is no 
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research on the subject of teacher evaluation at DLIFLC. This study therefore provides 
data to fill this gap in the literature. 

Purposes of Teacher Evaluation 
The first theme to be reviewed addresses the purposes of teacher evaluation. 
Teacher evaluation was identified as being for the purposes of quality assurance and 
professional development (Barber, 1985; Beerens, 2000; Cardno, 1999; Crumbley, Henry, 
& Kratchman, 2001; Doyle, 1983; Duke & Stiggins, 1986; Ellett & Teddlie, 2003; 
Iwanicki, 1990; Natriello, 1990; Stodolsky, 1990). JCSEE (2009) acknowledged that 
personnel evaluation is a difficult task and that unfortunately it is often performed poorly. 

JCSEE (2009) indicated that the standards are provided to facilitate the building 
of sound personnel evaluations. By adhering to the personnel standards, evaluation 
systems might be improved. Although the personnel standards are not specific to teaching, 
the standards are relevant and applicable to teacher evaluation. Particular attention was 
paid to the utility standard because it is directly related to evaluation for the purpose of 
growth and development. A brief description of the standards is provided and applies to 
both purposes of teacher evaluation. 

The 27 personnel standards are grouped into four categories or attributes. The 
four essential attributes identified are: propriety, utility, feasibility, and accuracy: 
The propriety standards ensure that evaluations are legal and ethical. 
• The utility standards provide for the timeliness and usefulness of the 

evaluations. They require "that evaluations have a constructive orientation and 
provide information useful to individuals and to groups of educators for 
improving their performance" (JCSEE, 2009, p. 6). One example of the utility 



23 

aspect is to ensure evaluators are well-trained to conduct the evaluations. In 
particular, this standard contains a topic that is directly related to the study: 
Follow-up and Professional Development. 

• The feasibility standards recommend that evaluation systems be efficient and 
easy to implement. 

• The fourth and final set of standards are the accuracy standards. These 
standards contribute to the validity of evaluations by encouraging, for 
example, that an adequate number of data sources be obtained to provide a 
complete view of performance. 

JCSEE (2009) suggested that the standards will produce sound evaluations that 
will contribute to greater satisfaction for those who are evaluated as well as those who 
evaluate. These standards are applicable to the DLIFLC teacher-evaluation system. The 
study examined the effectiveness of the various teacher-evaluation strategies and 
processes as related specifically to one of the four personnel standards — the utility 
standard — which addresses teacher growth and development, among other aspects. 

Teacher evaluation is categorized as being either formative or summative. 
Numerous definitions abound for these two types of teacher evaluation; however, it 
should be noted that these definitions have not changed significantly over the years. For 
example, JCSEE (2009) defined formative evaluation as "evaluation conducted while an 
evaluatee is performing; designed and used to promote growth and improvement in the 
evaluatee's performance or in a program's development" (p. 195). Cangelosi (1991) 
offered the following definition of formative evaluation: "An evaluation of instruction is 
formative if its sole purpose is to provide information that is useful for decisions about 
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how to teach" (p. 12). In contrast to formative evaluation is summative evaluation, which 

JCSEE (2009) defined as "an evaluation designed to present conclusions about the merit 

or worth of a person's performance" (p. 197). The comprehensive definition of 

summative evaluation of instruction follows: 

An evaluation of instruction is summative if it is a judgment of instructional 
effectiveness that is used for purposes other than helping the teacher decide how 
to teach. Unlike formative evaluations, summative evaluations may influence 
administrative decisions about that teacher (e.g., regarding retention, salary, or 
promotion). (Cangelosi, 1991, p. 13) 

The important distinctions between these two types of evaluation merit elaboration. 
Formative evaluation is discussed first. 

Formative Evaluation 

Formative evaluation serves the purpose of developing and improving teachers 
(Zepeda, 2006). Indeed, it is the goal of formative evaluation to provide teachers with 
feedback that will lead to improvement in teacher performance. Formative evaluation is 
for professional development. The evidence obtained during formative assessment will 
help structure conversations regarding professional development. Formative evaluation is 
needed to promote professional learning. Danielson (2008) astutely highlighted that the 
need for improving teaching exists because teaching is difficult, and although a lesson 
may be successful, it can always be improved in some aspect. Formative evaluation data 
can be used as feedback and a teacher may decide, based on the feedback received, to 
change a particular teaching practice (Peterson, 2000). 

Over the years, numerous researchers have agreed that the first and main purpose 
of teacher evaluation is for teacher development and improvement of teacher 
performance (Beerens, 2000; Cardno, 1999; Crumbley et al., 2001; Doyle, 1983; Duke & 
Stiggins, 1986; Iwanicki, 1990; Natriello, 1990; Stodolsky, 1990). Doyle (1983) asserted 
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that evaluation for teacher development can assist in both personal and professional 
growth. Danielson and McGreal (2000) wrote that formative judgments are a necessary 
part of teachers' learning and growing processes. Self-evaluation and portfolio 
development, for example, are described as formative-evaluation strategies (Airasian & 
Gullickson, 2006; Zepeda, 2006). Beerens (2000) specifically argued for the importance 
of teacher involvement in professional development. Beerens posited that teacher 
involvement is critical to long-term success and offered specific examples of 
professional-development opportunities that include study groups, action research, and 
peer coaching, among others. 

The literature highlights the importance of the formative aspect of evaluation. 
Centra (1993) clarified, "Formative evaluation is used to improve teaching performance; 
the information is given to teachers, whether it is obtained from students, colleagues, or 
faculty development specialists, and is meant to bring about positive changes" (p. 5). 
Danielson and McGreal (2000) agreed that formative evaluation is for "the purpose of 
enhancing the professional skills of teachers" (p. 8). They further elaborated that 
formative evaluation involves "providing direction for staff development" (Danielson & 
McGreal, 2000, p. 8). Additionally, it should be noted that it is not only the teacher who 
has responsibility for self -improvement in the teaching arena; institutions are also obliged 
to provide resources and information to faculty who desire to improve their performance 
(Duke & Stiggins, 1990; L. D. Fink, 2002). Centra (1993) further asserted that it is best to 
implement formative strategies so that teachers can become familiar with the procedures 
and criteria that will be used and have an opportunity to improve their performance prior 
to being evaluated or judged. 
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Formative-evaluation processes play a significant role in the growth and 
development of teaching faculty. For faculty to learn and grow professionally, they must 
receive feedback and there must be indications of how well teachers are performing. 
Schools may require teachers to achieve final outcomes or develop products. The 
products or outcomes reveal the progress or accomplishments made by the teacher. For 
example, teachers may be asked to prepare portfolios, present analyses of taped classes, 
or conduct action research projects. Formative outcomes might also consist of teachers' 
reports regarding completed university programs or continuing-education courses. 
Evidence of the teachers' experiences with the formative-assessment process should be 
considered. For example, teachers may be asked to give an oral presentation or submit a 
narrative report of what they learned. The teacher-supervisor discussions of the teachers' 
professional development should encourage an exchange of ideas for further 
collaboration with colleagues. Discussions regarding the teachers' experiences should be 
kept in the formative area and not be considered part of the teachers' formal or annual 
evaluation (Danielson, 2008; Danielson & McGreal, 2000). 

At DLIFLC, department chairs who supervise teachers and evaluate them 
formatively are asked to assist teachers by providing them with clear, objective, and 
constructive feedback, to serve as coaches, and to listen to their teachers' concerns 
regarding classroom issues or professional-growth topics. This study examined the 
perspectives of teachers and administrators regarding the effectiveness of the feedback 
process in the DLIFLC teacher-evaluation program. 
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Summative Evaluation 

The second purpose of evaluation is for quality assurance or accountability, 
rendering a judgment as to whether a given teacher is a good teacher. Researchers have 
concurred that teacher evaluation may also be used for administrative purposes and 
include such reasons as allocating faculty resources and determining tenure or salary 
(Crumbley et al., 2001; Stodolsky, 1990). Crumbley at al. (2001), who examined 
university students' perspectives on faculty evaluations, reported that "evaluations are 
used for making personnel decisions, allocating faculty resources, diagnosing and 
improving teaching performance, and choosing courses and instructors" (p. 204). Duke 
and Stiggins (1990) acknowledged that accountability "has tended to dominate the 
thoughts and actions of school officials charged with the responsibility for teacher 
evaluation" (p. 116). Accountability has indeed been an important trend in higher 
education according to Schuck, Gordon, and Buchanan (2008), who wrote that today 
higher education is plagued with "greater accountability, budgetary constraints and 
devaluing of the teaching profession" (p. 537). They further noted that "Checklists and 
easy-to-digest numbers are used to describe both teaching quality and professionalism" 
(p. 538). These researchers challenged the assumption that improved scores on student- 
satisfaction surveys are evidence of student learning. They viewed the prevalence of, and 
emphasis on, student-satisfaction surveys as a view of accountability that does not 
enhance professionalism. 

In a discussion on summative evaluation, Shinkfield and Stufflebeam (1995) 
documented the significance of an important report, A Nation at Risk, published in 1983 
by the National Commissions on Excellence in Education. This important report focused 
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on the problems related to teacher performance. The document stimulated increased 
attention to accountability in education. The public and those involved in education 
scrutinized the goals and outcomes of education, which resulted in "increased guidelines 
and standards to assess teacher evaluation systems" (Shinkfield & Stufflebeam, 1995, 
p. 23). Shinkfield and Stufflebeam acknowledged that greater importance was given to 
summative approaches: outcomes took on greater importance than teacher development. 
Subsequently, the focus on heightened accountability has continued into the 21st century. 
Kyriakides and Demetriou (2007) reported that the two general functions of an evaluation 
system are accountability and improvement. They clarified that accountability is 
summative in nature and that "it reflects the need to determine the competence of 
teachers in order to ensure that services delivered are effective" (p. 46). Danielson (2001) 
concurred by reporting that quality assurance is the first and only purpose recognized by 
policymakers; those entrusted with public funds are responsible for ensuring that 
classrooms are staffed with competent teachers. 

Teacher evaluation for the purpose of accountability encompasses a variety of 
aspects. For example, Iwanicki (1990) reported that teacher evaluation for the purpose of 
professional growth results in school improvement; and the improvement of schools is 
directly related to accountability or quality assurance. According to Iwanicki, 
accountability serves to ensure that incompetent teachers do not continue in the 
classroom. Natriello (1990) maintained that teacher evaluation serves the purpose of 
aiding in selection, retention, and in the removal of teachers. Barber (1985) specifically 
addressed accountability by positing that teacher appraisal serves to protect students from 
incompetence and also protects teachers from unprofessional administrators. Haefele 
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(1993) also addressed accountability, describing the purposes of evaluation related to 
accountability as serving to (a) screen out teachers who are ineffective as well as to 
identify outstanding or excellent service, (b) provide evidence that will withstand legal 
scrutiny, and (c) help institutions remove incompetent teachers. 

Barber (1985) observed that educational research has confused the formative and 
summative aspects of personnel evaluation. Barber contended that "too many people ask 
too many things of educational personnel evaluation systems, and too few people 
understand the ramifications of using one evaluation system for so many different 
purposes" (p. 4). This author argued that the two functions of teacher evaluation, that is 
improvement versus reward and/or punishment, should be separated. By separating 
formative from summative evaluation, schools would be better able to reduce the 
suspicion and mistrust that teacher evaluation has caused. 

Similarly, Milanowski (2005) highlighted the conflict between the summative and 
formative aspects of performance evaluation, indicating that the summative role generally 
overshadows the formative role of the supervisor or administrator. Milanowski explained 
the arguments for separating the two types of evaluation. By creating a formative system 
that has no summative consequences, the evaluatees would be less defensive, more 
willing to set challenging goals, and more open about the areas they need to improve. 
Also, the evaluator would be able to allow more time for the process of development and 
coaching. When summative evaluation is emphasized, evaluators may not make the time 
to analyze and discuss problem areas and suggestions for improvement (Milanowski, 
2005). 
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Moreover, summative evaluation encourages comparisons among people; whereas, 
formative evaluation emphasizes focusing on the individual's strengths and weaknesses. 
Seldin (2006) confirmed that although formative evaluation examines differences within 
individual teachers, summative evaluation examines differences between individuals. 
This study investigated the views of DLIFLC foreign-language teachers and 
administrators regarding the objective of teacher evaluation. Participants were asked to 
indicate their degree of agreement regarding improving performance, which is contingent 
on formative feedback, as an important objective of the evaluation process. They also 
were asked to report their degree of agreement regarding assessing quality of instruction, 
the summative aspect of the teacher-evaluation process, as an important objective of the 
evaluation process. 

There seems to be a general consensus in the literature regarding the various 
purposes of teacher evaluation. Most controversy or debate regarding teacher evaluation 
surfaces in the discussions of how the data or feedback on teachers or their teaching are 
collected and used. Teachers and administrators agree that evaluation of teaching is 
important and necessary, but they have had questions regarding the procedures used and 
how the evaluations were conducted. They also have expressed concerns regarding the 
lack of effect or impact on teachers (Danielson & McGreal, 2000; Timperley, 1998). 
Finally, Danielson (2001) emphasized that the formative and summative aspects of 
teacher evaluation can be merged to provide effective teacher evaluation. Accountability 
aspects of summative evaluation are rigorous, valid, reliable, and legally defensible; and 
the administrative purpose of evaluation was characterized as being "tough" (Danielson, 
2001). The formative process, which is designed to promote learning, is described as 
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being collaborative and supportive. It also is characterized as being "softer." Danielson 
(2001) further recommended that teacher evaluation systems respect teacher 
professionalism and involve teachers in activities that lead to growth and development. 

The DLIFLC teacher-evaluation system includes both the growth and 
development and accountability aspects of teacher evaluation. Supervisors responsible for 
evaluating foreign-language teachers must provide feedback to help the teachers grow in 
their profession and improve their performance. DLIFLC supervisors must also conduct 
annual, summative appraisals of their faculty members for the purpose of quality 
assurance. 

Problems with Teacher Evaluation 

Teacher evaluation or teacher appraisal is a complex phenomenon (Cardno, 1999) 
and therefore, arouses debate and controversy. Some researchers argued that teacher 
evaluation has become a pro forma process. They posited that it rarely influences teachers' 
classroom performance and consequently, may not be of value to teachers (Danielson & 
McGreal, 2000; Marshall, 2005). Other problems identified in the research literature 
concerning teacher evaluation are the perceived lack of training of administrators who 
evaluate teachers and the dual roles of administrators who serve as coaches during 
formative assessment and as judges during summative evaluations, among other issues. 

Department chairs at DLIFLC who supervise and evaluate foreign-language 
teachers are offered training in conducting classroom observation for professional 
development. Supervisors are encouraged, but not required to take this training. Those 
who do attend this training learn effective strategies for collecting objective evidence of 
classroom performance when conducting classroom observation. They also obtain 
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information regarding how to conduct meaningful pre-and postobservation discussions 
with teachers. There is not, however, training on how to use data from student 
evaluations of teaching or how to use data from other evaluation strategies when 
conducting formative and/or summative evaluations. DLIFLC undergraduate deans, who 
supervise foreign-language department chairs, may discuss data collection and use of the 
data in performance evaluations with supervisors; however, there is no evidence as to 
whether these discussions occur. This presents a problem for the Institute since 
supervisors may not fully appreciate the importance of using multiple sources of data. 
Nor is it known if these supervisors understand how to use the data to help teachers grow 
professionally and to improve performance. This study examined foreign-language 
teachers' and administrators' views on the two problem areas related to teacher 
evaluation that were identified in the research literature: the lack of administrator training 
in conducting teacher evaluation and the dual roles of the supervisor. 

Administrator Training 
One of the problems identified in the literature on teacher evaluation deals with 
the lack of training for administrators in data-collection and data- analysis procedures. 
Another issue is that evaluators also lack training in preparing programs for teacher 
improvement or professional development (Cardno, 1999; Raths & Lyman, 2003). 
Cardno (1999) specifically commented on the problem of evaluators' ineffectiveness in 
improving teaching due to the difficulty of the task and the conflicting role of evaluators 
as advisors, in the formative process, and supervisors or judges, in the summative process. 
Cardno also explained that student teachers are oftentimes coached by experienced, 
"cooperating" teachers who are not trained as faculty-education specialists. These more 
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experienced classroom teachers assigned to coach the student teacher may have received 
a booklet containing a set of guidelines, but they may not have read the guidelines. The 
experienced classroom teacher simply has not been professionally trained to formatively 
coach or evaluate the student teacher. Painter (2000) also wrote about the need for 
administrator training to help them become skilled evaluators. Painter reported that 
administrators rated themselves high in their abilities to evaluate teachers; however, they 
rated themselves lower in the area of developing a plan to help teachers who needed to 
change teaching practices. 

Beerens (2000) described experiences as a new principal, acknowledging that the 
administrative team in the district was not trained in teacher evaluation and that there was 
no reliability or consistency. Beerens explained that some of the schools had a high 
percentage of "superior" teachers and that some teachers received inflated ratings; 
whereas, other schools had a preponderance of "average" teachers. Beerens described the 
disgruntlement of the teachers as a result of the inequities of the evaluation ratings, and 
suggested that evaluators or administrators should be trained to develop processes and 
activities that teachers will find meaningful and that will affect teaching so that the goal 
of increased student performance is achieved. 

Administrators and other educational personnel tasked with evaluating faculty are 
often unaware of what is involved in ensuring that they are conducting effective 
evaluations of faculty. Seldin (2006) stated that they find themselves in alien territory 
when it comes to evaluations of teaching performance. Well-trained evaluators must be 
able to make reliable judgments based on evidence (Danielson & McGreal, 2000). 
Training for evaluators should include information on effective procedures for data 
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collection. Evaluators will also need to know "which data points are relevant to each of 
the evaluative criteria" (Danielson & McGreal, 2000, p. 74). Finally, interpreting the 
evidence, judging it based on the descriptions of the performance levels of the criteria, 
and assigning an overall rating are all important aspects that require carefully designed 
training. 

Danielson and McGreal (2000) provided a list of relevant topics important for 
effective evaluator training. An effective training program on teacher evaluation should 
ensure that evaluators fully understand the complete teacher-evaluation system. The 
training program should include the following information on how to (a) use the 
evaluative criteria, (b) reduce bias in observations and analysis of teaching, (c) use 
evidence rather than opinion, (d) evaluate teacher performance, and (e) use the evaluation 
forms and related instruments. Seldin (2006) suggested that evaluators should also be 
trained in how the results of the evaluation will be used. Training should include 
information on how the teacher evaluations will lead to professional development (Seldin, 
2006). 

Some training is provided to DLIFLC supervisors who evaluate foreign-language 
teachers; however, the training does not address all of the evaluation strategies that are 
used in teacher evaluation. This study specifically asked foreign-language teachers and 
administrators to indicate their level of agreement regarding supervisors having been 
adequately trained to evaluate teachers' performance. 

Dual Roles of Administrators in Teacher Evaluation 

Also problematic in teacher evaluation are the dual roles of administrators or 
directors in providing both formative and summative appraisals (Cardno, 1999). Some 
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researchers have emphasized that formative and summative approaches to evaluation 
need to be separated (Beerens, 2000; Harris, 1986). Peterson, Wahlquist, and Bone 
(2000) questioned whether administrators who evaluate teachers can be both 
"instructional leaders and summative judges" (p. 136). Centra (1993) declared "People 
involved in formative evaluation — whether chairs, colleagues, or faculty development 
specialists — will not be as effective in helping teachers if they are also making 
summative judgments based on the information obtained" (p. 5). Centra further reported 
that teachers will not be open and will not feel comfortable discussing their teaching 
questions or weaknesses with someone who will also evaluate them. 

Harris (1986) recommended that a priority be given to developmental teacher 
evaluation. It must be remembered that the purposes of teacher evaluation should be tied 
to the type of evaluation. For example, when evaluating for accountability, a summative 
evaluation is required; whereas, for professional development, a formative evaluation is 
needed. Le Maistre, Boudreau, and Pare (2006) described a 4- year, longitudinal study 
that used semistructured interviews lasting 40-60 minutes. The researchers interviewed 
the graduate students and their supervisors. Included among the participants were 
students and supervisors in education, social work, and other helping professions. The 
interviews were recorded, transcribed, and analyzed from various perspectives. 

The researchers examined the roles of university faculty educators who mentored 
and evaluated university students engaged in field experiences. The researchers posed 
critical questions as they assisted supervisors in conducting evaluations: "Are we 
evaluating appropriately? What values are reflected in these forms and are they what we 
value in new professionals? Are we evaluating what is needed to become professional? 
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Or do we evaluate what is easy to evaluate?" (Le Maistre et al., 2006, p. 348). Le Maistre 
et al. (2006) found that conflict and tension arose when the supervisor was both the 
mentor and evaluator and their research showed that supervisors disliked conducting the 
summative evaluations. They described the conflict experienced by supervisors who 
served as both mentors and evaluators in professional-training programs. These 
researchers advised that if the same person is assigned to be both mentor and evaluator of 
someone being guided into a profession, special training should be provided that would 
include guidelines regarding how to manage the dual roles. 

Milanowski (2005) provided an extensive review of the research literature 
regarding the conflict between the administrative and developmental aspects of 
performance evaluation and summarized the arguments for separating the two aspects of 
teacher evaluation. First, because formative evaluation has no summative consequences, 
the evaluatee would be less defensive and more honest about weaknesses, more willing to 
set challenging goals, and more inclined to be open to feedback. Second, by separating 
the roles, the evaluator would have more time to concentrate on the coaching process and 
giving feedback. Milanowski also suggested that when the emphasis is on summative 
evaluation, there is little time or effort given to examining the potential causes of 
performance problems and providing suggestions for improving performance. 

The study conducted by Milanowski (2005) took place in a large, Midwestern 
urban school district and involved 144 newly hired teachers. Fifty of the teachers were 
assigned to a split-role group. That is, they were assigned a mentor, an experienced 
teacher, who received professional training to carry out mentor duties; and they also were 
assigned to be evaluated summatively by an administrator and a peer evaluator. The 
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remaining 94 teachers were assigned to a combined group. They were assigned to an 
experienced teacher who served as both mentor and evaluator to the new teacher. The 
study employed multiple methods of data collection, including semistructured interviews 
and surveys. 

The results of the study noted no major differences between formative and 
summative evaluations conducted by the same person or formative and summative 
evaluations conducted by two separate individuals. A key finding of this study was that it 
may be more important to provide developmental assistance to the teacher than to have 
the roles of coach and evaluator be split. Although the conflict between summative and 
formative aspects of teacher evaluation has been documented in the literature, this study 
demonstrated that evaluator' s dual roles can be reconciled. A key finding was that it is 
important for the evaluator to provide developmental assistance for the evaluation to be 
effective. 

Chow et al. (2002) reported interesting findings from their study which was 
conducted in Hong Kong. They surveyed 550 elementary teachers in Hong Kong 
regarding whom they preferred to evaluate them for formative and summative purposes. 
Although these teachers expressed that their peers or senior teachers were more 
competent than the principal as an evaluator, they nevertheless preferred that their annual 
evaluations be conducted by their principal and that their formative appraisal be 
conducted by a peer. Most interesting is the mention they made regarding the fact that 
two national studies conducted in the United States showed that teachers in the United 
States also preferred that their principal conduct the summative appraisals rather than 
colleagues or other evaluators. Many principals in the United States, according to Chow 
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et al. (2002), are participating only in summative appraisals to "keep their administrative 
authority as the judge" (p. 97) and encouraging their teacher colleagues to conduct the 
formative evaluations. This study related to the DLIFLC evaluation system because 
foreign-language teachers are only evaluated by their department chairs, who supervise 
the teachers for both summative and formative purposes. 

Raths and Lyman (2003) pointed out the unavoidable conflict of interest when 
mentors or coaches make summative evaluations of their student teachers. Teachers in 
training may share their concerns or questions regarding aspects of their teaching with 
their principals or supervisors to later discover that these concerns have been documented 
as weaknesses in their summative evaluations. Another important factor related to the 
dual roles of administrators is that principals and other administrators want to avoid 
conflict and the feelings of discomfort that come with identifying poor teaching 
performance. 

Having only one person be responsible for both the formative and summative 
evaluations causes conflicts of interest and may result in a lack of trust between teacher 
and administrator (Beerens, 2000). Acheson and Gall (1997), who described clinical 
supervision, agreed that supervisors face a conflict in their dual roles as evaluators and 
facilitators, and highlighted that some educators argue for a separation of roles. They 
suggested that there is a "sting" in evaluation because the supervisor, who initially offers 
to assist the teacher to develop professionally and improve performance, is the same 
person who then prepares the summative appraisal for accountability purposes. They thus 
recommended that the supervisor's classroom observations be augmented by peer 
observations, which provide nonjudgmental feedback. 
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The role conflict that administrators experience when they work with teachers is 
well-documented. Danielson (2008) concurred that administrators wish to be guides for 
their faculty; however, they are also the teachers' judges. The conflict is exaggerated 
when dealing with newly hired teachers who are vulnerable to the evaluation made by the 
administrator. A new teacher will most likely be hesitant to ask the administrator any 
questions or share any concerns regarding teaching issues because the administrator will 
be conducting the annual evaluation on the teacher. Administrators must work hard to 
assure teachers that the formative discussions will not carry over into the annual appraisal. 

DLIFLC educational administrators, who evaluate foreign-language teachers, 
have much in common with other educational administrators who evaluate teachers in K- 
12 or higher education settings. That is, DLIFLC supervisors who evaluate teachers must 
oftentimes be both coaches who help teachers grow and develop to improve their 
teaching performance as well as judges who conduct summative appraisals. This dual 
role assigned to the evaluator of teachers can contribute to a tension that may hinder the 
growth and development of the teacher. This study examined the views of DLIFLC 
foreign-language teachers and administrators regarding the use of the evaluation 
processes to help teachers improve their teaching ability. 

Strategies used in Teacher Evaluation 

This study examined teachers' and supervisors' perspectives regarding the 
effectiveness of various teacher-evaluation strategies in promoting the teachers' 
professional development. It is, therefore, important to describe these strategies. The third 
theme of this review addresses teacher-evaluation strategies. There are a variety of 
strategies used to evaluate teachers. Some of the most popular strategies used are 
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(a) classroom observation, (b) SET, (c) portfolios, (d) action research, and (e) self- 
evaluation. It is important to note that whereas each strategy may provide valuable data 
for evaluation, multiple sources of data should be considered for effective teacher 
evaluation (Stronge et al., 2008). Zakrajsek (2006) wrote that these strategies provide 
data that can be used to improve teaching performance, which can enhance student 
learning. 

The evaluation strategies listed below are those most commonly described in 
research studies. These evaluation strategies are also used at DLIFLC to varying degrees. 
Each of the teacher-evaluation strategies is described and, where possible and relevant, 
examples of the strategies found in research studies are included. 

Classroom Observation 

The first and most common form of teacher evaluation to be examined is 
classroom observation. It is reported to be one of the oldest evaluation processes in 
education and is considered to be important to teacher development (Harris, 1986). 
Classroom observation also is acknowledged to be the mainstay of the majority of 
teacher-evaluation systems. It is described as involving direct observation of a teacher 
performing in a classroom. Most classroom observations are conducted by an 
administrator such as a principal or a department chair who serves as a supervisor, but 
they can also be conducted by other teachers or even students. Another hallmark of 
classroom observation, as a tool for collecting data for teacher evaluation, is that the 
observer normally uses some type of observation instrument (Darling-Hammond, Wise, 
& Pease, 1983). The previous description of classroom observation, documented over 2 
decades ago, is still fairly accurate today. Darling-Hammond et al. identified some of the 
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limitations of using observation data as evidence for teacher evaluation. They posited that 
limitations include observer bias, insufficient sampling of performance, and poorly 
designed instruments that can threaten validity and reliability. 

Sullivan and Zirkel (1999) referred to classroom observation as a traditional tool 
used by administrators to evaluate classroom performance. They described classroom 
observation as a process in which principals visit the teacher's classroom, take notes 
during the observation, and then have a conference in which the observations and 
suggestions for improvement of performance are discussed with the teacher. Linking any 
observed deficiencies to performance objectives is also mentioned as an important part of 
the process. Allwright (1988) wrote that systematic observation served as a mirror for the 
teachers to see their behaviors with clarity and thus be prompted to change. Similarly, 
Peterson (2000) wrote about "systematic observation," described as a process in which an 
observer records teacher performance and then analyzes the data based on a conceptual 
framework. 

Peterson (2000), in contrast, affirmed that not all classroom observations qualify 
as systematic observations. Peterson and Kauchak (1982) set forth guidelines or criteria 
that reflect the minimal requirements for observations to be considered to be systematic 
observations in teacher evaluation: 

The observer is trained in observation techniques and is monitored for 

reliability. 

• Observations are based on a reliable number of visits — six to eight 
unannounced visits are suggested. 

• Observations have a limited focus. 
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• Recording systems, whether checklists or narratives, are systematic and 
reliable. 

• Data are analyzed based on an agreed upon conceptual framework that is 
linked to student learning. 

Peterson (2000) cautioned that failure to apply the above guidelines threatens the 
fairness, accuracy, and credibility of systematic observation, and that classroom 
observation is normally conducted by a principal or other administrator. Peterson 
concurred that it is the most common form of teacher evaluation. The advantages of 
classroom observation are reported to be that it is effective in capturing evidence 
regarding classroom climate, teacher-student rapport, and interaction. Also, observational 
data are useful as formative information when observers present evidence by means of 
transcripts or observation notes for the teacher to consider for improvement (Peterson, 
2000). 

Danielson and McGreal (2000) argued that classroom observation is the best way 
to capture essential aspects of teaching. They also clarified that not all areas of teaching 
can be evaluated through observation. For example, feedback teachers provide to students 
is best appraised by examining student work such as reports or tests that contain the 
teachers' feedback to the student. Stodolsky (1990) also discussed the weakness in only 
using classroom observation for evaluative purposes: "Classroom observation leaves out 
direct systematic evidence about teacher planning, teacher assessment and modification 
of instructional materials" (p. 178). 

Although acknowledging that classroom observation may not be the only model 
that should be used in effective teacher-evaluation systems, Acheson and Gall (1997) 
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confirmed that clinical supervision, "has as its goal the professional development of 
teachers, with an emphasis on improving teachers' classroom performance" (p. 1). They 
reported that clinical supervision is a model that requires both supervisor (the observer) 
and teacher (the observed) to work together as participants who analyze and interpret data 
and make decisions collaboratively. They described clinical observation as being a 
democratic, teacher-centered process that involves three-steps: (a) the planning 
conference, (b) the classroom observation, and (c) the feedback conference. Each of these 
steps is vital to the process, which works in a cyclical fashion. That is, at the end of the 
feedback conference, the teacher and supervisor discuss future plans for further planning 
conferences and observations. 

McGreal (1983) stressed the importance of what was referenced as the first stage 
in clinical supervision, the preplanning stage. McGreal clarified that one of the important 
features of this stage is to establish a relaxed relationship between supervisor and teacher 
and to know the teacher's goals. In this stage the teacher presents an overview of the 
lesson and the supervisor may make suggestions or ask questions to assist the teacher in 
developing the lesson. It is suggested that a narrow focus for the observation be 
established. Danielson and McGreal (2000) suggested that it is in this initial phase that 
the reliability and usefulness of the observation are determined. With respect to the 
second stage of the process, the observation, McGreal (1983) stressed that the 
observation notes should be descriptive and not evaluative. It is during this stage that the 
data are analyzed by the evaluator. The observation is a time when data are collected as 
evidence for use in professional discussions and reflection on teaching and learning 
(Danielson & McGreal, 2000). Finally, in the third stage, the teacher examines what 
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transpired during the class based on the data, and with the help of the evaluator, the 
teacher analyzes and interprets the data. Danielson and McGreal (2000) emphasized that 
the postobservation conference is a time for reflection, constructive feedback, and 
guidance or reinforcement for the teacher. 

During the postobservation conference or shortly after the conference, the teacher 
decides on alternative approaches for the future, considering the satisfactory and 
unsatisfactory aspects of the lesson. Lastly, the feedback conference for one observation 
may become the planning conference for a subsequent observation. An important point is 
that the clinical supervision model is based on the assumption that teachers, when given 
objective data, will be able to analyze, interpret, and act in self-directed and constructed 
ways. McGreal (1983) reported that the stages of classroom observation have also been 
referred to as clinical supervision, stressing that in clinical supervision the emphasis is on 
a close, collegial relationship between teacher and supervisor. The focus of clinical 
supervision is on teacher motivation and improvement rather than on accountability 
(McGreal, 1983). 

It should be noted that, whereas many researchers include classroom observation 
as a viable strategy for teacher evaluation, there are those who have expressed skepticism 
regarding the validity of teacher evaluations based solely on administrators' observation 
reports (Peterson et al., 2000). For example, Shulman (2004) reported that classroom 
observations demonstrate the full complexity of teaching; however, unfortunately 
observations of teaching have not captured many of the critical dimensions of teaching. 
Problems identified with classroom observation for teacher evaluation include the fact 
that generally generic rating scales are employed. For example, the same criteria on a 
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rating scale are used to assess an elementary-level class and a high school class. 
Oftentimes observers have not had the proper training to evaluate the teaching they 
observe. Training in observation methods and techniques is critical to effective data 
collection (Shulman, 2004). In this regard, Harris cautioned, "Untrained observers cannot 
see adequately, fail to control biases, do not sample events, and produce inadequate 
records" (1986, p. 131). Observing involves data collection and does imply judgment and 
evaluation (Harris, 1986). 

There is agreement in the research literature that traditional classroom 
observations continue to serve as one of the principal sources of evaluating teachers. This 
statement holds true for classroom observation at DLIFLC, where the study was 
conducted. Classroom observations are one of the most common teacher-evaluation 
strategies used at DLIFLC for both formative and summative evaluations. Moreover, 
foreign-language department chairs who supervise and evaluate foreign-language 
teachers are required to conduct a minimum of four classroom observations for each 
teacher they supervise. These department chairs receive training provided by the DLIFLC 
Faculty Development Division on how to collect observable data and how to conduct the 
observation conferences. This study examined participants' perspectives on the 
effectiveness of classroom observation as a teacher-evaluation strategy. 

Student Evaluation of Teaching (SET) 

SET, in addition to class observations, is one of the main sources of data for 
teacher evaluation. SET is perhaps as popular if not more popular than classroom 
observations and is characterized as being vast and used excessively in personnel 
evaluations (Barth, 2008; Pallett, 2006; Pounder, 2007). In fact, Seldin (1999) 
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specifically stated, "student ratings have become the most widely used source of 
information to evaluate teaching" (p. 15). According to Centra (2003), a considerable 
amount of research has been amassed regarding SET. Of all the methods used to evaluate 
teaching, student evaluations are the most popular, with over 2,000 references to SET in 
the ERIC system. Cashin (1999) additionally reported that there "probably are more 
studies of student ratings of teaching than all of the other data used to evaluate teaching 
combined" (p. 25); in 1999 there were 2,175 references to student evaluation of teaching. 

According to Lord (2008), the practice of asking students to evaluate their 
teachers began in 1927 at Purdue University when students in a sociology class were 
asked their opinions of the course. The survey data were read solely by the professors for 
self-improvement and were not shared with the administration. Lord explained that it was 
in the 1960s that students discovered the power they had in higher education. It was 
during the period of free speech and resistance to the war in Vietnam that student 
evaluations of instructors became more oriented to student satisfaction than to a teacher's 
instructional effectiveness. 

Seldin (1993, 1999) affirmed that student ratings at some institutions are often the 
only method of obtaining information on teacher effectiveness. In fact, reports indicate 
that student ratings have increased in use from 54.8% in 1978 to 88% in 1998 (Seldin, 
1999). It is also acknowledged that student ratings can be an excellent source of data for 
formative or summative evaluations if used properly and as one of multiple sources of 
data. As popular as SET is, it must be noted that SET is not without criticism (Cross & 
Steadman, 1996; Ngware & Ndirangu, 2005; Seldin, 1999). 
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The research literature shows that SET is often used as the sole measure of 
teaching performance and is controversial (Peterson et al., 2000). Some researchers have 
pointed out that SET can be compared to customer- satisfaction surveys and researchers 
claim that little feedback about teaching or course effectiveness is obtained from SET 
(Beyers, 2008; Oliver-Hoyo, 2008; Titus, 2008). The student-as-consumer metaphor is 
described in the literature as one in which learning is viewed as a product or commodity 
rather than as a process (Rowley, 2003; Titus, 2008). Other researchers have argued that 
SET is a popularity contest and that the ratings do not contribute to new knowledge 
delineating effective teaching (Emery, Kramer, & Tian, 2003; Finch, Helms, & Ettkin, 
1997). For example, Beyers (2008) outlined numerous examples of problems with 
student surveys and claimed that student evaluations tell teachers things they know are 
incorrect or untrue. Beyers also commented that teachers oftentimes do not receive credit 
from students for things they did well, and teachers are not always held accountable by 
students for things they did poorly. Beyers provided an anecdote to exemplify the issues. 
Based on firsthand observations of university students filling out surveys about their 
teachers, one professor observed students giving little attention to the evaluations. Further, 
those students who tried to give thoughtful attention to their answers on the survey were 
pressured by their classmates to finish. 

Additionally, there exists the unfortunate notion that in order to receive more 
favorable student ratings, some teachers lower their grading standards and reduce the 
workload (Beyers, 2008; Lord, 2008). Centra (2003), however, found no evidence of 
improvement in evaluations of teachers who were lenient in assigning coursework and in 
grading. Beyers (2008) pointed out that "students are not generally versed in pedagogy, 
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so their evaluations often have more to do with their emotional experience than with 
learning" (p. 106). 

According to Pounder (2007), SET is used more than other forms of teacher 

appraisal, yet there is disagreement regarding the value and significance of SET scores 

and their relationship to student achievement. Pounder concluded that 

although the SET system has its advocates, by and large, most studies have called 
into question the value of the SET system. It seems that there are so many 
variables unrelated to the actual execution of teaching influencing SET scores that 
they tend to obscure accurate assessment of teaching performance. (Pounder, 
2007, p. 186) 

Crumbley et al. (2001) reported that SET is questionable when it is used to make 
personnel decisions. SET has further been characterized as being (a) inappropriate for 
measuring instructional effectiveness, (b) demoralizing in faculty morale, (c) cheap and 
convenient because student-opinion questionnaires are easy to administer and score, and 
(d) easy to abuse when used as the only source of data when evaluating teachers (Emery 
et al., 2003; Ngware & Ndirangu, 2005; Seldin, 1993; Titus, 2008). The research 
literature on SET further revealed that it is not appropriate to ask students about the 
relevance of course materials or how well the teacher knows the material being taught, as 
students are not qualified to evaluate instruction (Lord, 2008; Rowley, 2003; Seldin, 
1993). 

Another argument against the use of SET deals with the question of the validity 
and reliability of student questionnaires. In the United States and Australia, the validity of 
student-rating forms may be in question because teacher-evaluation instruments are often 
tied to tenure and promotion. Questions are raised regarding the validity, accuracy, and 
relevance of student feedback questionnaires. Rowley (2003) reported that insufficient 
attention is given to the design of the questionnaires. This researcher contended that users 
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of student-feedback questionnaires should be aware of the limitations surrounding this 
teacher-evaluation strategy. Rowley recommended that student feedback be triangulated 
with other sources of evidence. 

Another concern regarding SET is that once the student assessment of the teacher 
in question is completed, there are no mechanisms provided to assist teachers in their 
desire to improve and develop their teaching effectiveness (Finch et al., 1997). Beerens 
(2000) referred to teacher evaluation as a "dead end" because in many programs a link 
has not been established between teacher evaluation and teacher development. Many 
faculty members view high student ratings of teaching as mere popularity contests, 
whereas others view low student ratings as marks of rigorous and demanding standards in 
their teaching (Finch et al., 1997). The principal problem with student ratings is that for 
some educational programs it is the only source of input for teacher appraisal. There is 
consensus in the research literature that good teacher-evaluation systems use multiple 
sources of data or feedback (Beerens, 2000; Finch et al., 1997; Ngware & Ndirangu, 
2005; Peterson, 2004; Seldin, 1999). 

Crumbley et al. (2001), who reported that use of SET for personnel decisions was 
questionable, also observed that students will punish teachers on the ratings for 
embarrassing students when the students have not done their homework or have not 
prepared for class. SET data must be reliable and neutral, and it must be noted that SET 
data may lead to grade inflation. Also, if SET data are made available to students, they 
will select the courses where they will receive a higher grade. 

Numerous researchers (Centra, 1993; Peterson & Kauchak, 1982; Strange & 
Ostrander, 2006; Wilkerson et al., 2000), on the other hand, indicated that the research on 
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the subject of SET reveals that this type of data is valid, reliable, and useful in improving 
teaching. SET is also recognized to be an excellent source of information because 
students 

(1) know their own personal situation well, (2) have closely and recently observed 
a number of teachers, (3) uniquely know how pupils think and feel, (4) directly 
benefit from good teaching, (5) report in numbers that foster high reliability (in 
the 0.80-0.90 range), (6) furnish relatively inexpensive and unobtrusive 
information, and (7) are stakeholders and consumers of good teaching. (Peterson 
et al., 2000, p. 137) 

The survey study conducted by Peterson et al. (2000) assessed the views of 9,765 
K-12 Davis County School District, Utah students regarding teacher performance. The 
study described a teacher-evaluation system that allowed teachers to select student ratings 
as one of the data sources for their evaluations. Student surveys were selected by 401 K- 
12 teachers as a source of data for their evaluation. The researchers reported the 
descriptive statistics including mean, standard deviation, kurtosis, and skewness for each 
survey item. Additionally, they conducted interitem correlations and factor analysis. They 
found that student evaluation of teaching obtained via student surveys provided valid and 
reliable data. Reliability for all scale items for the three levels was reported as 0.766 
elementary level, 0.820 middle school level, and 0.917 secondary level. These 
researchers asserted that student surveys are not popularity contests; and they credited 
students with understanding the distinction between liking a teacher and recognizing the 
teacher's ability to teach. The study included a teacher-satisfaction survey and the 
findings were that teachers responded favorably to the student surveys. 

Peterson et al. (2000) asserted that the positive responses of teachers and 
administrators who participated in the study were important, given the fact that there is 
evidence in the literature that teachers are dissatisfied with evaluations based solely on 
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administrator reports. The researchers further advised that student views of teacher 
performance are important and are accurate indicators of performance. Al-Issa and 
Sulieman (2007) reported their findings based on their study of university students' 
perceptions of evaluations of teaching. These researchers stated that students responded 
favorably to having the opportunity to evaluate their teachers. They further reported that 
the results of their study showed that student evaluations are biased by the grade a student 
received from the teacher, the teacher's age, gender, and nationality. They, therefore, 
concluded that their findings cast doubt on the validity of SET. 

Despite the controversy over student evaluation of teaching, it has become one of 
the most used sources of teacher evaluation. Oliver-Hoyo (2008) emphasized that student 
evaluations can help improve teaching, but that they do not necessarily need to be a 
measure of the quality of teaching. Peterson (2000) acknowledged that it is important to 
use multiple sources of data; teachers should have the option to include student surveys in 
their evaluations. 

SET, along with classroom observation, is one of the most frequently used 
teacher-evaluation strategies at DLIFLC. Students are asked to evaluate teachers' 
classroom performance multiple times during the course of their studies at DLIFLC. 
Foreign-language department chairs do not receive any formal training on how to use 
student evaluations of teaching as a tool to help teachers improve performance. The 
general practice is that department chairs provide the teachers with student comments and 
discuss how the comments might be used to inform the teacher's practice. The instrument 
used in this study collected information from teachers and administrators regarding their 
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views on the effectiveness of SET and whether this evaluation strategy contributes to the 
teacher's professional development. 

Portfolios 

This section describes the third key approach to teacher evaluation, which is 
through the use of portfolios. The professional literature abounds with descriptions of the 
use of portfolios for teacher evaluation. It is argued that portfolios for teacher evaluation 
make a contribution to more sound personnel decisions and to the growth and 
professional development of teachers (Seldin, 1997; Wolf, 2006). It is important to begin 
the discussion on portfolios with a definition. Following the definition, an explanation 
will be presented as to how portfolios are related to teacher evaluation. 

Portfolios are descriptions of a teacher's strengths and accomplishments, which 
can include documents and materials used in teaching and which can also be used for 
teacher improvement (Seldin, 1997). The best portfolios reflect teachers' 
accomplishments and include evidence of both student and teacher learning. They 
additionally can contain captions and commentaries that describe the contents (Shulman, 
2004; Wolf, 2006). An effective portfolio will include examples of student work as well 
as examples of the teachers' instruction and evaluations. Portfolios were further described 
as evaluation and professional-development tools that provide tangible evidence of 
classroom instruction and, when used for teacher improvement, can include but are not 
limited to the following specific examples of artifacts: (a) summaries of teaching methods, 
(b) course syllabi, including lesson plans and student assessments, and samples of student 
achievements related to the objectives, (c) samples of innovative teaching practices, 
(d) student ratings of the teacher and of the course(s), (e) research articles, and 
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(f) classroom observation reports (Danielson, 2008; Danielson & McGreal, 2000; Seldin, 
1997). 

Wolf (2006) claimed that teaching portfolios are tools that are used today for both 
evaluation and professional development. Zepeda (2002) provided an example of how 
portfolios may be used to link evaluation to professional development. As part of the 
evaluation and supervision process, a 2-year case study was conducted at an elementary 
school. The purpose of the project was to link portfolio development to clinical 
supervision and to teacher evaluation. The principal of the elementary school invited the 
researcher to assist with the problems identified, which were that (a) the school's current 
evaluation system did not contribute to the teachers' professional development, and 
(b) teachers were not active participants in their professional development. Twenty two 
teachers participated in the voluntary project of preparing a portfolio that would be tied to 
the clinical- supervision process, including observations and pre- and postobservation 
conferences. The results were that teachers reported a number of benefits related to the 
portfolio-development process, such as enhanced reflection on their teaching. 

Portfolios may be used in formative evaluation, to improve teaching performance, 
and in summative evaluation for personnel decisions (Attinello et al., 2006; Seldin, 1997). 
When a portfolio is prepared for formative purposes, it should contain more evidence of 
reflective analysis, which reveals the teachers' thought processes regarding how the 
teachers might adjust their teaching. For example, teachers might discuss different 
materials, techniques, and objectives they will include in future teaching sessions. When 
a portfolio is used for summative purposes, it can include some of the same content that 
is found in portfolios used in formative evaluations, but will also include student ratings 
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and administrator's assessment reports of the individual's teaching contributions. 
Murphey and Yaode (2007) described portfolios as having the potential to greatly impact 
teacher development. They also emphasized that there is no training method that can be 
as relevant to teachers as identifying their strengths and building on them. With respect to 
portfolios as a source for summative evaluations, the portfolio assessment system must be 
valid, reliable, fair and free from bias (R. S. Johnson, Mims-Cox, & Doyle-Nichols, 
2006). 

Portfolios may be developed using a formal and directed process or an informal 
and self-directed process. In the formal and directed process, teachers receive guidelines 
regarding the portfolio's contents. In the informal and self-directed process, teachers are 
free to select the contents. Effective portfolio programs also have specific assessment 
criteria and give the portfolio preparer the freedom to select the appropriate contents. It is 
also important to note that portfolios may be developed by teachers as well as 
administrators (Martin-Kniep, 1999). 

Attinello et al. (2006) reported that teacher evaluation has remained unchanged 
since the early 20th century and described teacher evaluation as a time-consuming ritual. 
They contended that due to recent educational reform there is a renewed interest in 
teacher evaluation, and stated that one way teacher-evaluation systems have been 
modified is that portfolios have been added to the assessment systems. Attinello et al. 
described a survey study they conducted with 798 teachers and administrators. The 
participants were asked to comment on the value of using portfolios for measuring 
teacher performance and for professional growth. The participants reported that they 
believed portfolios were accurate depictions of teacher performance, but that portfolios 
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did not reflect all aspects of teaching. Following the collection of the survey data, the 
researchers conducted focus-group and personal interviews. Some participants found 
portfolios to be time consuming and their value was questionable. 

Chapman, Pettway, and White (2001) described a newly developed evaluation 
system that consisted of student surveys, self-evaluation forms, peer-evaluation forms 
and class- specific forms. After implementing these evaluation tools, a program portfolio 
was implemented. Chapman et al. discussed the reasons for using portfolios as part of 
teacher-evaluation systems. Included among these is the fact that portfolios present hard 
data and evidence regarding teaching quality and student learning, and having hard data 
is critical in times of budget cuts and other economic realities. Secondly, they noted that 
the selection of items to be included in the portfolio allows the preparer the opportunity 
to think critically about teaching. Thirdly, the teaching products placed in portfolios can 
be shared with new teachers. These materials can be valuable to a new teacher who will 
benefit from seeing another teacher's materials, lesson plans, and student tests. 

Finally, portfolios serve the function of professional development and a means for 
self -reflection. Danielson (2008) suggested that portfolios are appropriate for both novice 
and experienced teachers. However, they are particularly relevant for tenured teachers 
because oftentimes they are only evaluated every 3 or 4 years. Additional benefits of 
using portfolios for teacher evaluation included the fact that constructing one's own 
portfolio can boost the preparer's self-esteem, and motivation (Murphey & Yaode, 2007). 
R. S. Johnson et al. (2006) posited that portfolios encourage evaluatees to move from a 
passive role in evaluation to an active role by engaging in more complex thinking and 
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self-evaluation. Another benefit of using portfolios is that the items and artifacts selected 
may be used at a future time for tenure and promotion considerations. 

One issue of considerable disagreement in the literature addresses the question of 
whether well- assembled portfolios can gloss over ineffective teaching. Seldin (1997) 
argued that they cannot, positing that a portfolio preparer cannot document effective 
teaching practice if the evidence is not there. Seldin emphasized that glossy covers and 
computer graphics will not suffice and will not disguise weak teaching performance. In 
contrast, Attinello et al. (2006) found in their study that some teachers and administrators 
expressed concerns about the accuracy of portfolios. That is, these teachers and 
administrators believed that ineffective teachers might compile excellent portfolios. 
Attinello et al. further found that administrators believed that they could tell when a 
portfolio was not an accurate portrayal of the teacher's classroom effectiveness. 
Administrators in the study also conducted classroom observations as part of the 
evaluation system, which allowed them to see teachers' instructional performance, hence 
their strengths and weaknesses. 

Another area of concern related to portfolio development and the assessment of 
portfolios is that both processes are time consuming and labor intensive. It takes time to 
assemble a valid, professionally organized portfolio, and a good deal of time is also 
required for administrators to thoroughly review the portfolio (Attinello et al., 2006; R. S. 
Johnson et al., 2006; Wolf, 2006). The study conducted by Attinello et al. included 
recommendations to ensure effective use of portfolios as part of an evaluation system. 
They suggested that teachers and administrators receive appropriate training to ensure 
that clear guidelines for portfolio implementation are given and understood by all. 
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Chapman et al. (2001) described the training workshops they held to help teachers with 
portfolio preparation. Workshops were given that included explanations and examples of 
portfolio assembly. Discussions and assignment of readings of scholarly articles were 
included. Finally, Chapman et al. provided recommendations to those interested in 
implementing portfolios as a means of assessment. For example, they recommended that 
those interested in implementing a portfolio program allow time for planning and 
research, be selective in the contents of portfolios, and consult with other departments 
that may already be using portfolios. 

Portfolios have become an important strategy for teacher evaluation. They should 
be used as part of a multidimensional evaluation system that includes multiple classroom 
observations with feedback and support as well as other evaluation strategies (Attinello et 
al., 2006; Danielson, 2008). Attinello et al. also concurred with previously mentioned 
research that claimed that portfolios can be used for the purpose of accountability as well 
as for professional development. 

Portfolios are not commonly used as a teacher-evaluation strategy at DLIFLC; 
however, given the number of researchers who espouse the benefits of portfolios, it was 
beneficial to find out the views of foreign-language teachers and supervisors regarding 
the possible effectiveness of this tool as a teacher-evaluation strategy. 

Action Research 

This section provides a description and a discussion of action research as a 
strategy for teacher evaluation. A review of the scholarly literature on action research 
revealed that action research is used at all educational levels, K-12 as well as in higher 
education. DLIFLC also incorporates action research. In fact, it is incorporated into the 
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performance standards for classroom teachers who hold the ranks of Associate Professor 
and Professor. Teachers at other ranks are encouraged to conduct research; however, it is 
a requirement for those at the higher ranks. 

Action research is identified as an assessment strategy that promotes inquiry 
through self -reflection and engages teachers in examining their instructional practices 
(Glanz, 1999; Sagor, 2000; Zepeda, 2006). It is also described as a problem-posing and 
problem- solving process that can be used to solve educational problems and to make 
individual and school improvements (Gebhard & Oprandy, 1999; Tomal, 2003). Zepeda 
(2006) claimed that action research is one of numerous data sources regarding teacher 
performance and assessment. Wallace (1998) wrote about action research and evaluation 
in the context of language teaching, observing that evaluation is at the heart of the action 
research philosophy and that it primarily consists of a self-evaluation of one's work. 
Wallace highlighted the normal tendency to negatively evaluate oneself, which leads to 
motivation to improve one's practice through action research. 

Action research, thus, is identified as a valuable data source for teacher growth 
and development and in this way is relevant to formative processes of evaluation. Action 
research consists of steps that are cyclical and consists of the systematic collection and 
analysis of data related to an aspect of teaching for the purpose of reflection and 
application. It allows teachers the opportunity to investigate events in their own 
classrooms, to take the necessary steps toward solving the problem or question, and to 
reflect on the results (Wallace, 1998). The action-research process is described as 
consisting of a series of repeated steps that form a reflective cycle: (a) identification of a 
question or problem, (b) data collection, (c) data analysis and interpretation, and 
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(d) taking action and reporting results (Allwright & Bailey, 1994; Gebhard & Oprandy, 
1999; Glanz, 1999; Nunan, 1992; Sagor, 2000). Reporting the results of the action 
research is an important aspect of this evaluation strategy, which leads to professional 
development. 

For example, Taber (2007) recommended that sharing action research may be 
done in informal groups, in a department of a school, or the results may be published in a 
professional journal. A. Burns (1999) provided a number of options for reporting on 
action research, suggesting that the practitioner can present a workshop or create visual 
displays such as video or photographic displays. A. Burns also encouraged reporting at 
poster sessions that can be presented at professional conferences. The telling of the action 
research story, otherwise known as reporting outcomes or results, whether oral or written, 
is an important part of contributing to the profession. 

Peterson (2000) contributed to the definition of action research by stating that it is 
conducted by practitioners using local resources, related to curriculum design, which is 
advantageous as a unique data source for evaluation that can highlight innovative 
instructional materials and tests. Peterson also considered that action research can be a 
disadvantage in that it requires more work on the part of the teacher, compared to other 
teacher-evaluation strategies such as student ratings, administrator evaluations, and 
observations. Further, Peterson (2000) commented that action research creates additional 
work for the evaluator. 

Bray (2007) conducted an action-research study demonstrating that evaluation is 
part of a cycle that moves from action to reflection. Bray contended that evaluation 
should focus on professional development rather than summative measures, and viewed 
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teacher evaluation as embedded in professional development. In this action-research 
study Bray observed and evaluated English as a Second Language (ESL) teachers who 
were teaching literacy skills to migrant students in Australia. The author explained that 
the process used in conducting the action research was to involve ESL teachers by 
interviewing them and discussing issues with them. Bray clarified that the action-research 
project conducted in this study demonstrated that it was conducted by the practitioners, 
was collaborative, and focused on changing something. 

Through the action-research process that included workshops, interviews, and 
extended discussion, the researcher was able to successfully evaluate both the teachers 
and the framework they were implementing. Bray (2007) further clarified that by 
collaborating with teachers, evaluators can encourage teachers' professional growth. That 
is, the researcher worked closely with the teachers, by observing them as they 
implemented the multiliteracy framework in their classes and also by interviewing them 
regarding their teaching experiences with the framework. Another way in which the 
researcher and the teachers collaborated is that the teachers were asked to choose and 
apply part of the framework. 

Action research as a teacher-evaluation strategy serves to motivate teachers to 
grow as professionals. It can provide teachers with opportunities to gain insight into their 
teaching performance and their roles as educators. Action research includes two 
important characteristics that are important for effective teacher evaluation and 
professional growth: it is self-directed in that the teacher identifies the problem or 
question and it includes collegial interaction when results or findings are shared (Beerens, 
2000). This study asked DLIFLC teachers and administrators to share their views on the 
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effectiveness of action research as a teacher-evaluation strategy. It was instructive to 
learn what the perspectives of the participants are; that is, to specifically learn if DLIFLC 
teachers and supervisors believe that action research helps teachers grow and develop. 

Self-Evaluation 

The fifth and final teacher-evaluation strategy to be reviewed is self-evaluation. 
As previously mentioned, in traditional teacher evaluation, the teacher is observed by an 
administrator and a report of the class visit is then presented to the teacher. Oftentimes, 
the observation is prefaced by a pre- and postobservation conference. In this traditional 
system, the teacher is passive, whereas the administrator is actively involved in 
conducting the observation, taking notes, and providing feedback to the teacher. Airasian 
and Gullickson (1997) offered a definition of self-evaluation that highlights teachers' 
active involvement. They defined self-evaluation as a process that encourages teachers to 
make judgments about the effectiveness of their knowledge and performance for the 
purpose of self -improvement. Danielson and McGreal (2000) suggested that evaluation 
should go beyond observation and include teacher self-assessment as part of teacher 
appraisal: 

Teachers are highly perceptive of their own skills in teaching, and are, or can 
become extremely accurate in their perceptions. Further, the act of self- 
assessment requires reflection, an activity documented through research to yield 
professional learning and growth. Self-assessment may be incorporated into either 
the formal evaluation process, or a process for self-directed professional growth 
or both. As a source of information, self-assessment can add much to the dialogue 
regarding the quality of teaching. (Danielson & McGreal, 2000, p. 47) 

Beerens (2000) confirmed that teachers who are the most effective are also those 
who are best at using their self-assessment skills; and Schlecty (1990) referred to self- 
evaluation as "the most powerful form of evaluation" (p. 115). Braskamp and Ory (1994) 
argued that self-assessment is important for self -reflection. In a discussion regarding the 
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documentation of teachers' professional activities, Peterson (2006a) wrote that teachers 
have a responsibility to self-assess and maintain their own records of professional 
development. Teacher self-evaluation also provides useful information for planning and 
teacher improvement (Danielson, 2008; Peterson, 2000) and is, therefore, considered to 
be an important data source for teacher-evaluation systems. 

In discussions of teacher self-evaluation, reference is made to self -reflection. For 
example, Braskamp and Ory (1994, p. 102) wrote, "self-assessment involves reflection 
and judgment." These researchers emphasized that teachers are the most important 
assessment source because they are the most knowledgeable, compared to other data 
sources, regarding their work and the rationale behind their work. They further 
recommended that schools should foster a climate in which teachers are consistently 
monitoring and assessing their own progress. Bullard (1998) also discussed the 
importance of "self (reflective) evaluation" (p. 9). Bullard described several methods that 
allow teachers to self-assess and reported that when teachers think about what they do 
and why they do it, they are practicing reflective thinking. The reflections that teachers 
have help them grow as professionals. 

Darling-Hammond et al. (1983) also wrote about faculty self-evaluation as an 
approach to teacher evaluation that encourages reflection. These researchers indicated 
that the combination of self-evaluation and goal- setting may contribute to self-reflection 
and motivation to change and grow. When teachers consider all of the available data 
sources such as student ratings, for example, they make judgments about their own 
teaching. Darling-Hammond et al. clarified that self-evaluation is not suitable for 
summative evaluation; however, it can be used for formative evaluation and professional 
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development. Danielson and McGreal (2000) viewed self-assessment as an important part 
of a teacher evaluation system because it promotes professional learning. "A teacher 
evaluation system, then, committed to maximizing the professional growth of teachers, 
should include a focused approach to structured reflection on practice" (p. 48). 

Related to self-reflection and evaluation, Wood (1992), an administrator at an 
independent high school, wrote about a narrative-based evaluation program conducted 
with teaching narratives, which included teachers' reflections of their learning and 
teaching experiences. Wood reported dissatisfaction with traditional teacher-evaluation 
strategies such as clinical supervision and student ratings, especially when working with 
experienced teachers. Limitations of the traditional evaluation strategies were that they 
were too mechanical and that they were better suited for novice rather than experienced 
teachers. 

Wood (1992) interviewed four teachers who volunteered to participate in an 
alternative evaluation plan and described their reflections, which consisted of stories 
related to their teaching. Wood observed that the experienced teachers needed to study 
their own experiences in order to improve their practice. The alternative teacher- 
evaluation program was designed to contribute to the experienced teachers' professional 
development. The new, voluntary evaluation system, included interviews between the 
teacher and the supervisor, a reflection by the teacher with accompanying goals, a 
classroom observation and feedback, a written self -evaluation by the teacher, and a 
written evaluation by the supervisor. 

Wood (1992) provided cautionary notes regarding the model for faculty 
evaluation. First, the model may not be suitable for beginning teachers. Interviews 
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between the teacher and the administrator should be focused on teaching practice, 
whereby the teacher is a reflective practitioner. The approach was time consuming. A 
checklist approach would have been more efficient; however, it would not have 
encouraged reflection by the teacher, whereas reflection is precisely what experienced 
teachers need: "By putting teachers' experiences at the center of the evaluation process, 
however, supervisors provide a vehicle to help teachers view their work seriously as 
grounds for theory-making, problem- solving, and decision making" (Wood, 1992, p. 549). 

Brandt (2007) conducted case-study research in which an innovative self- 
evaluation strategy was used in training English as a Foreign Language/ESL teachers. 
This self-evaluation strategy incorporated reflection. In this research study, the 
investigator implemented a strategy that involved participants recording their self-talk 
during the training course. The recordings were transcribed and served as the basis for 
written narratives. The narratives served as evidence of the trainees' reflection and 
assessment of their work in the course. The study found that participants solved problems 
by the personal process of reflection and evaluation. Brandt described several caveats that 
emerged as a result of the experience. First, formative and summative evaluations are 
often in conflict; and therefore suggested that self-evaluation is a formative activity. 
Second, the process of recording and then converting speech to written form was time 
consuming and might not appeal to all. Finally, the narratives were only one of various 
strategies available to encourage reflective practice including keeping a journal or a 
portfolio. The narratives contributed to the trainees' self-evaluation and could also be 
used with experienced teachers as part of their ongoing professional development. 
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Edstrom's (2006) self-evaluation is particularly relevant to this study in light of 
the fact that the study was conducted in a foreign-language classroom, similar to this 
study. Edstrom conducted a personal study of use of first language in the foreign- 
language classroom. There were 15 participants, Spanish 101 students, who filled out 
questionnaires regarding Edstrom's use of English in the researcher's classrooms. The 
study contained a description of the data analysis and findings. In the study, the 
investigator engaged in reflection, critical thinking, and analysis by recording and 
transcribing the classes and keeping a journal of reflections and insights. As part of the 
study, Edstrom compared personal perceptions to those of the students and found that 
techniques of self-recording and analyzing the classes were effective for self-evaluation. 
Edstrom noted that the goal was to learn about the researcher's own teaching, become 
more aware of what the researcher said in the classroom, and to improve the researcher's 
teaching effectiveness. Further, the researcher hoped the process of reflective evaluation 
might be stimulating for other language teachers. The research conducted by Edstrom 
provided a perfect example of a self-evaluation topic that a DLIFLC teacher might 
consider studying. 

Bahous (2006) described a study based on a self-assessment form developed and 
conducted with 18 K-12 teachers in Lebanon. Bahous analyzed the teachers' responses to 
a self-assessment questionnaire developed to determine teachers' perspectives of their 
innovativeness, how they motivate their students, and their involvement in reading 
professional journals. Additional benefits to teachers' self evaluation were that when 
teachers self-evaluate they reflect on their actions and methodologies and become more 
convinced of their need to improve their performance; and when teachers engage in self- 
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evaluation, they are more willing to change and will more readily participate in 
professional development than if it were directed by others. 

Lord (2009) described experience with self-evaluation at an institution of higher 
education, reporting that at the beginning of the year, teachers met with their department 
chair and a review committee to discuss objectives and activities for the year. At the end 
of the year, teachers prepared a report providing their rationale for being continued or 
promoted and submitted various items such as syllabi, lesson plans, and assessments 
developed for the courses taught, as part of their appraisal. Teachers also included student, 
peer, and administrator evaluations. Faculty members were also encouraged to submit 
artifacts related to scholarship and service. Following the submission of items and reports, 
materials were reviewed and the appraisal was filed in the department as well as in the 
teacher's personnel file. 

Lord (2009) recalled that the process was fair and nonthreatening. One of the 
major benefits of self-evaluation was that, unlike other forms of teacher evaluation, this is 
one method in which the teacher is always invested. The evaluation process described by 
Lord is similar to the DLIFLC teacher-evaluation system. DLIFLC foreign-language 
teachers also meet with their department chairs to discuss the performance standards and 
objectives for the year. At the end of the year, DLIFLC teachers submit their "input," a 
type of self- assessment. Teachers are free to incorporate reference to student comments 
from student evaluations of their teaching. 

Airasian and Gullickson (1997, 2006) discussed teacher investment when they 
outlined eight reasons why teacher self-evaluation is an important process for teachers as 
part of formative evaluation. They posited that teacher self -evaluation 
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1 . Is a professional responsibility. 

2. Focuses on professional development and improvement. 

3. Recognizes that organizational change happens when individuals change 
themselves (not from hierarchical directives). 

4. Gives teachers voice. 

5. Makes teachers aware of the strengths and weaknesses of their practice. 

6. Encourages teacher development. 

7. Treats the teacher as a professional. 

8. Encourages collegial interactions. 

Each of the abovementioned reasons contributes to teachers' involvement and 
professional growth. 

Ross and Bruce (2007) conducted an explanatory case study in which they 
concurred that self-evaluation is linked to professional development. Based on their 
findings, these researchers claimed that "self-assessment is a powerful technique for self- 
improvement" (p. 147). This qualitative study included 10 mathematics teachers who 
were given a self-assessment tool that, when combined with other professional-growth 
strategies such as peer coaching, observation, and an in-service program, contributed to 
the teachers' professional growth. The researchers described their methodology and 
presented a model of teacher self-assessment as a mechanism for change. 

One area of concern regarding self-evaluation in teacher evaluation is that 
teachers' perspectives of their teaching effectiveness do not necessarily agree with 
student ratings. In fact, according to Peterson (2000), there is little agreement between 
teachers' self -ratings of their teaching effectiveness and ratings reported by students, 
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coworkers, and administrators. An important consideration was made by Centra (1973), 
who stated that teacher self -ratings were recommended as a potential source for 
performance improvement; however, teacher self-ratings would not have much validity 
when considered for personnel decisions such as promotion or salary. Centra (1973, 
1993) further reported that teachers tend to exaggerate their performance as teachers and 
cited statistics that indicated a low correlation between teachers' self-evaluations with the 
evaluations of students, colleagues and administrators. Because teachers' self-evaluations 
may not be accurate, there is an argument against using them in summative evaluations 
(Centra, 1993; Peterson, 2000). Centra (1993) further summarized the reasons why self- 
evaluation should not be used in summative evaluations. Self-evaluations (a) do not 
provide meaningful measures of teaching performance, (b) lack validity and objectivity, 
and (c) can become a justification for not improving. Centra (1993) did agree, however, 
that teacher self-evaluations are relevant to instructional improvement. 

Self-evaluation involves reflection and analysis. According to Airasian and 
Gullickson (2006) in order for self-evaluation to lead to change, teachers must have 
awareness of their teaching practice. Additionally, teachers must be committed to their 
practice and to their responsibility to understand, evaluate, and improve this practice. It is 
important to acknowledge that when self -evaluation is combined with additional 
evaluation strategies, it can be used for formative as well as summative purposes. Given 
the strong recommendation and benefits of using self-evaluation as a teacher-evaluation 
strategy, it was beneficial to learn what teachers and administrators think of the 
effectiveness of this evaluation strategy. 
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Supervision and Leadership in Teacher Evaluation 
The fourth theme of this literature review consists of a discussion of supervision 
and leadership in the context of teacher evaluation and professional growth and 
development. In most cases, it is a supervisor or administrator who is responsible for 
evaluating teachers. Therefore it is critical, as was mentioned previously, that these 
individuals be knowledgeable and skilled in implementing a teacher-evaluation system. 
As Danielson and McGreal (2000) explained, "The process of evaluating teaching is a 
complex cognitive task" (p. 74) and evaluators have much to consider when evaluating 
teacher performance. The complex task of serving as judge, in summative evaluation, and 
as coach, in formative evaluation, becomes even more complex when considering the 
recommendation that teacher-evaluation systems be based on multiple sources of data 
(Beerens, 2000; Finch et al, 1997; Peterson, 2004; Seldin, 1999). 

In addition to skillfully managing the various data sources as an evaluator of 
performance, the supervisor is also a leader, someone who is responsible for motivating 
and guiding teachers in their professional development. This section focuses on 
leadership practices of the supervisor or leader by addressing the role of trust as a 
leadership principle, as well as the importance of performance feedback for teacher 
growth and development. DLIFLC department chairs who serve as supervisors who are 
responsible for evaluating foreign-language teachers must strive to establish trusting 
relationships with their teachers in order to provide the context for performance feedback 
that is appreciated by the teachers. 
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Trust as a Leadership Principle 
Beerens (2000) argued that an effective evaluation system must incorporate 
successful leadership principles. One of these leadership principles is trust; followers 
must trust the leader. In school settings where teacher-evaluation systems are 
implemented, trust between leaders and teachers is critical to the success of the 
assessment program. When leaders who serve as evaluators demonstrate that they have 
integrity and that they are honest, teachers will be more likely to trust them. Teacher- 
evaluation scholars affirmed that when teacher evaluation takes place in an environment 
that encourages mutual trust between the teacher and school leader, there is potential for 
the school and especially the students to benefit (Stronge et al., 2008). Goleman (1998) 
stressed the importance of trust to have an effective coaching relationship, viewing the 
role of coaching as a major skill for developing others. Covey (1991) emphasized that 
leaders who are trustworthy and competent will be better able to engage in productive 
relationships that are characterized by clear communication, empathy, synergy, and 
interdependency. 

Trust was also identified as a professional value that must be upheld by 
educational supervisors. Holland (2004) reported on an ongoing study of five 
administrators who were interviewed and shadowed at their schools over a 3-year period. 
The five administrators answered open-ended questions regarding their experiences as 
administrators working with teachers in their schools. Holland studied how these 
administrators practiced supervision and how they upheld their managerial and 
professional values. 
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In the discussion regarding professional values, Holland (2004) explained that 
trust was the most notable of the values and a necessary condition for judgment and 
interpretation, the other professional values. The administrators trusted in their own 
ability to perform their jobs as supervisors and they also trusted their teachers to make 
appropriate decisions regarding teaching and learning. Trust must exist between the 
administrators and the teachers (Acheson & Gall, 1997; Holland, 2004; Shinkfield, 1994). 
Holland highlighted the benefit of teachers and administrators who worked together 
collaboratively by forming small groups and who thus cultivated trust and respect for 
each other. Holland summarized the findings by reporting that the role the administrators 
played went beyond the evaluation of teaching; that is, these administrators worked with 
their teachers as fellow professionals who engaged in collaborative decision-making 
processes regarding teaching and learning. 

Ponticell and Zepeda (2004) reported on their 2-year study of 100 elementary and 
secondary school teachers who were enrolled in administrator-preparation programs. The 
study examined teachers' and principals' understanding of supervision and evaluation as 
viewed through the theoretical lens of symbolic interaction. Data were collected and 
analyzed based on essays, interviews, and focus-group notes. All teachers and the 
majority of the principals they interviewed equated supervision with evaluation. That is, 
rather than viewing supervision as a means for promoting teacher reflection and 
improvement in teaching, teachers and principals viewed supervision as a judgment or an 
evaluation. 

The teachers and principals who participated in the study agreed that supervision 
meant adhering to legal requirements. When evaluating teachers, principals in the study 
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fulfilled the steps that were required by law. The principals were described as taking 
hierarchical actions: observing, monitoring, and checking the teacher. Ponticell and 
Zepeda (2004) described numerous interactions between teachers and principals that 
demonstrated the lack of productive evaluation strategies such as a principal arriving late 
to observe a class or not appearing for a previously scheduled observation. In another 
example, a principal asked the teacher to fill out the evaluation form and sign it because 
the principal admitted to not having time to do an observation or complete the evaluation 
form. Ponticell and Zepeda (2004, p. 53) posed an important question: "If an effective 
evaluation system depends on trust between teachers and principals, what trust can be 
developed when teachers experience supervision and evaluation as an empty process or 
as retribution or manipulation?" 

Davis, Ellett, and Annunziata (2002) reported findings from a two-school, 
comparative case study that linked teacher-evaluation systems, school leadership, and 
improvement in teaching. The research suggested that school leadership, related to 
teacher evaluation, is important to improve teaching and learning. These researchers 
found that leadership made a positive impact on teacher evaluation, distinguishing 
between "perfunctory summative teacher evaluation and meaningful assessment of 
teaching and learning" (p. 288). Davis et al. investigated the role of leaders in two public 
schools and found the first school leader to be hostile and negative toward a newly 
implemented teacher-evaluation system. In the second school, they found a leader who 
was supportive of the new evaluation system. 

Teachers at the first school expressed fear and anxiety as a result of the 
implementation of the new system. The researchers also found no evidence of 
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professional growth, nor was there any evidence of training to assist the teachers with the 
implementation of the new evaluation system at this school. Teachers at the second 
school were found to be engaged in meetings in which they discussed teaching and 
learning based on the new evaluation system. Confidence and trust were also seen as 
contributing to the teachers' willingness to accept change. Davis et al. (2002) concluded 
by indicating that leadership is what makes the difference between having an evaluation 
system that is a meaningless obligation and one that is a meaningful process to improve 
teaching and learning. 

Calabrese, Sherwood, Fast, and Womack (2004) conducted a qualitative 
embedded case study, using the left- and right-hand-column case method. Eighty K-12 
teachers and eight principals participated in the study. All data were triangulated. 
Participants were interviewed; however, only the teachers were asked to write an 
imaginary summative-evaluation conference with their principal. The researchers found 
that the teachers were reluctant to return their imaginary summative evaluations to their 
principals. The researchers stated that the teachers' lack of trust toward their principals 
was reflected in the data. Calabrese et al. claimed that the principals and teachers had 
different purposes and perspectives for the summative evaluations. Principals were 
obliged to follow district policies regarding the evaluations, whereas, teachers wanted 
coaching for growth and development. Teachers viewed the evaluation process as being 
unfair and meaningless and principals viewed it as being useless and added that the 
evaluation rating scale was the biggest source of miscommunication. 

Zepeda (2004) wrote about trust and teacher growth in the context of a teacher- 
evaluation system. The researcher conducted a case study and reported on the efforts of a 
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newly assigned principal and assistant principal who changed a school's pro forma 
evaluation system, which had previously consisted of a single observation of teachers 
conducted at the end of the year. The principals used supervision to create a learning 
community. Senge (1990) referred to learning organizations, characterized as places 
where individuals discover how to create reality and how to change it. The two 
educational administrators in the study found that at the school they were assigned to lead, 
there was no connection between the processes of supervision, staff development, and 
teacher evaluation. These two leaders tied supervision and evaluation to the professional- 
development activities to encourage school improvement. 

The new principals implemented a peer-coaching program and established a 
schoolwide committee to work on linking professional-development activities to their 
school's improvement. Teachers were given the choice to participate in the peer-coaching 
system or to have the principal or vice principal conduct the four cycles of clinical 
supervision and prepare the year-end summative rating. Zepeda (2004) affirmed that the 
principals encouraged the creation of a community of learners and changed the school's 
leadership paradigm by instituting opportunities for school personnel to engage in 
"inquiry, generative problem-solving, dialogue, and reflection" (p. 146). Through the 
process of inquiry and dialogue, the teachers and principals learned to trust and have 
confidence in each other. Additionally, the school leaders further indicated that they had 
to let go of their hierarchical control to allow teachers to explore their own learning. In 
this way supervision and evaluation were recast to encourage professional development 
and growth. DLIFLC supervisors strive to establish collaborative relationships with the 
teachers they evaluate in order to gain their trust and respect. The trusting relationship 
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then serves as a first step toward promoting evaluation processes that lead to improved 
performance and professional growth. 

The Role of Performance Feedback in Supervision 

It is affirmed in the research literature that the role of a school leader is key to 
teacher evaluation, professional development, and student learning and achievement 
(Ovando & Ramirez, 2007). Whereas this important statement comes from the K-12 
research literature on teacher evaluations, it is also true for leaders at the DLIFLC who 
evaluate foreign-language teachers. One of the most important of all the responsibilities 
that rest with educational administrators or leaders is that of evaluating teacher 
performance (Shinkfield, 1994). Shinkfield suggested that the school principal or leader 
who is usually the person responsible for evaluating teachers is also responsible for 
providing constructive feedback to teachers. Feedback in teacher evaluation is viewed as 
essential in formative evaluation and has as its goal to improve teaching effectiveness and 
promote professional growth (Feeney, 2007; Shinkfield, 1994). 

Shinkfield (1994) additionally stressed the importance of the leadership role 
played by the principal in the teacher-evaluation process. The principal must understand 
the importance of the evaluation system. If a principal is not convinced of the usefulness 
of the evaluation, the staff will be skeptical and disillusioned. The principal must 
establish a positive climate regarding teacher evaluation and communicate to the teachers 
that the main purpose for the evaluation system is teacher improvement. Prior to 
reviewing the research on feedback on teaching performance, it is important to examine a 
definition of feedback. "Feedback is the information people receive about their 
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performance. It conveys information about behaviors and it conveys an evaluation about 
the quality of those behaviors" (London, 1997, p. 11). 

London (1997) explained that feedback can come from a variety of sources such 
as managers, coworkers, and subordinates. People are hesitant about being evaluated and 
about receiving feedback on their performance. Many individuals are also hesitant to give 
feedback. Feedback is, therefore, a delicate issue in organizations. Managers are hesitant 
to give negative feedback because they know it puts the receiver of the feedback on the 
defensive. Despite the difficulties in giving and receiving feedback, London contended 
that feedback is important in evaluating individual and group performance. 

London (1997) also outlined the benefits of feedback, positing that it directs 
behavior, influences future work goals, and increases employees' abilities to become 
aware of their own performance deficiencies, among other benefits. Feedback can be both 
constructive and destructive. Constructive feedback should be (a) clear — that is, it should 
be understood in the same manner by both the sender and the receiver of the feedback, 
(b) specific, (c) frequent and timely — giving feedback should be a common practice, 
(d) relevant, (e) from a credible source, (f) confidential, and (g) accompanied by an 
explanation that will help the recipient of the feedback know how to improve. Finally, 
London maintained that feedback is valuable because it directs and motivates behavior. 

Ovando (2005) concurred that principals play an important role in giving 
constructive feedback to teachers. Ovando conducted a qualitative study in which 27 
principals-in-training participated. Participants were asked to conduct walk-through 
observations of teachers and prepare feedback messages for the teachers they had 
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observed. The feedback messages were to contain areas of strength, areas of concern or 
questions for the teacher to think about, and suggestions for development. 

Upon conclusion of the field-based learning experience, the participants provided 
responses to open-ended questions and participated in a collaborative discussion 
regarding their experiences. Participants found that the teachers they had observed were 
appreciative of the feedback. Teachers reported that they valued the professional 
conversations and indicated that a sense of validation was achieved. An important 
limitation of the study mentioned by Ovando (2005) is that the aspiring school leaders 
were not the direct supervisors of the teachers they observed, and in actual practice the 
issue of authority may have an influence on teachers' reactions to the observations and 
the feedback. 

Several important lessons learned and recommendations were reported by the 
participants: 

• Teachers have a need for ongoing support in their professional growth and 
development. 

• Instructional leaders should conduct classroom observations frequently. 

• Feedback must be based on observable actions and must be specific. 

• Feedback should be linked to the teachers' professional development. 
Finally, Ovando (2005) recommended that in order for instructional leaders to be 
effective, they must have a strong knowledge base regarding quality instruction and 
classroom-observation techniques, and be able to provide clear and specific feedback to 
teachers. 
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Goleman (1998), who popularized the concept of emotional intelligence, also 
discussed the important role that feedback plays in developing others. Goleman's (1998) 
emotional-competence framework addresses personal and social competencies. 
Developing others is one of the competencies that comprise the social skills and is related 
to empathy, which the theorist described as reading the feelings of others. Kouzes and 
Posner (2002) commented that leaders who developed others found that their employees 
were more committed to doing their jobs. An organizational climate that encourages 
employee involvement and contributes to employees feeling important helps strengthen 
the employees' competence and confidence (Kouzes & Posner, 2002). People who are 
aware of others' development needs are, according to Goleman (1998), able to offer 
useful feedback and identify their employees' professional-development needs. 

Goleman (1998) discussed the role of trust and feedback in coaching relationships 
and clarified that coaching and mentoring help employees perform better. Goleman 
(1998) emphasized that a successful coaching relationship will be based on trust. 
Successful leaders serve as coaches who help their employees boost their performance by 
giving feedback and offering suggestions on developing the skills that the employee 
needs. Goleman suggested that respect, trustworthiness, and empathy were characteristics 
that helped employees grow and develop. Goleman (2001) also wrote that the ability to 
develop others is a hallmark of superior managers. 

Feedback is instrumental in the growth and development of teachers and also 
supports the accountability aspect of teacher evaluation. Most importantly, it is 
recognized as essential to the evaluation process and is documented as one of the 
standards in the Personnel Evaluation Standards: "Constructive evaluations provide 
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educators with information and professional feedback that build their professional self- 
knowledge, increase their enthusiasm, and enhance their efficacy as practitioners" 
(JCSEE, 2009, p. 70). Like principals or other educational supervisors who conduct 
teacher evaluations, DLIFLC supervisors are asked to provide developmental feedback to 
the teachers they evaluate. This study asks teachers and supervisors to provide their 
perspective on feedback in the evaluation process. 

Models to Facilitate Teacher Evaluation and Development 

There are numerous leadership models used in business, education, and other 
fields. The various models focus on different aspects of the role of the leader or 
supervisor. Two approaches will be identified in this section: transformational leadership 
and Danielson's (1996, 2008) framework for teaching. Each of these models facilitates 
the link between teacher evaluation and professional development. These models are 
important to this study because they offer guiding principles that can facilitate teacher 
evaluation. The first approach to be described is transformational leadership. 

Transformational Leadership 

J. M. Burns (1978) first developed the theory of transformational leadership, 
contrasting transformational leaders with transactional leaders, and claiming that 
transformational leaders motivated followers to go beyond their self-interest for the 
benefit of the group or organization. Bass (1985) expanded on the work of J. M. Burns 
(1978) and highlighted the roles and needs of the followers. Bass also emphasized that 
transformational leaders created supportive environments that allowed for trust to exist in 
relationships. Bass (1995) wrote that transformational leaders can be either directive or 
participative. Bass (1995) cited the example of Nelson Mandela who demonstrated 



80 

directive behavior when he told his countrymen to forget the past. Mandela, according to 
Bass, also demonstrated participatory leadership when he became involved in multiracial 
discussions and mutual agreements. Evaluators of teachers may also need to be both 
directive and participative, especially in light of providing performance feedback to 
teachers. 

Transformational leaders, according to Avolio and Bass (1995), will recognize 
individual differences and in this way participate in individualized consideration, one of 
the four behaviors that contributes to transformational leadership. Transformational 
leaders are transformed in that they look beyond task completion to the development of 
others. It is through the process of acknowledging individual differences that the 
transformational leader assists followers so that they develop to their potential and 
achieve higher levels of performance (Avolio & Bass, 1995). Individual consideration 
involves diagnosing followers' needs for professional growth and development and 
providing the needed coaching, which includes giving positive or negative feedback and 
mentoring to followers. Avolio and Bass further stated that it is not only the leader who is 
transformed, but also the follower. That is, the follower gains a broader perspective and 
moves from a self-interested perspective to understanding the contribution that can be 
made to the group or organization. 

Avolio and Bass (1995) outlined multilevel operational definitions of 
individualized consideration and provided examples of the definitions. At the individual 
level, each individual's needs are considered and equity rather than equality is 
highlighted. In the leader's behavior, followers' needs are diagnosed and more time is 
spent with new employees to ensure they are well-oriented to the job. At the team level, 
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individual differences are recognized by team members. An example of followers' 

behaviors is when team members provide each other with feedback related to team 

members' development. Finally, at the organizational level, there is understanding that by 

developing human resources, performance will be enhanced. Most importantly, at the 

organizational level, trust and respect are identified as core values by the organization. 

The organization demonstrates that it values the development of its employees. 

The behaviors related to individualized consideration that were identified by Bass 

(2008) reflect the behaviors needed to be carried out by evaluators of teachers: 

Individually considerate leaders pay special attention to each follower's needs for 
achievement and growth. New learning opportunities are created, along with a 
supportive climate. Individual differences in needs are recognized. The leaders 
serve as coaches and mentors for their followers. They attend to the individual 
followers' differential needs for growth and achievement. (Bass, 2008, p. 622) 

It is precisely these behaviors, reflective of individualized consideration that DLIFLC 
supervisors of foreign-language teachers need to demonstrate in order to facilitate the 
growth and development of teachers. DLIFLC department chairs who serve as 
supervisors of foreign-language teachers and who evaluate their performance will be 
more effective if they develop trusting relationships with their teachers. DLIFLC 
supervisors also will be more effective in helping their teachers grow and develop by 
acknowledging the individual differences of their teachers. 

Danielson 's Framework 
The second model or framework that can be used to link teacher evaluation to 
professional growth and development is Danielson's (1996, 2008) framework for 
teaching. The framework, according to Danielson (2008), is widely used for the purpose 
of evaluating teacher performance. Current evaluation processes are meaningless rituals 
of supervisory evaluation and the proposed framework can be used to transform teacher 
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evaluation into a process that promotes thoughtful examination of instructional 
excellence (Danielson, 2008). One important feature of the framework is that it offers a 
different approach to teacher evaluation for tenured teachers than it does for nontenured 
teachers. Danielson (2008) also proposed that the framework for teaching may be used as 
a framework for the supervisory process. 

Danielson (1996) identified a series of steps for implementing the framework. The 
first step, goal setting, consists of teachers conducting a self-assessment and reflection 
based on the framework. As a result of this self-evaluation, the teachers have identified 
areas that can be their focus. Additionally, the supervisor, based on observations or other 
sources of data, may also have identified areas for growth and development. The teacher 
and supervisor then meet to determine the goals, based on the domains and components 
of the framework that will reflect the teacher's needs. 

The second step in the process for implementing the framework for evaluation is 
to collect data. The teacher and supervisor, in addition to peer coaches or others, are 
asked to collect evidence that documents the teacher's performance in a particular 
domain and component. As an example, the teacher may collect materials developed, or 
student work, as evidence and incorporate this into a portfolio. The supervisor can 
incorporate evidence from classroom observations that reflect the teacher's progress in 
the identified domain and component. The supervisor and teacher should then meet to 
review and discuss the materials they have gathered to determine the teacher's 
performance and skill in the specific area identified. Self-assessment on the part of the 
teacher and feedback from the supervisor should be part of the performance-oriented 
conversation. 



83 

The third step identified by Danielson (1996) is the creation of a professional- 
development or improvement plan. Based on the goals established and the data reviewed 
in the performance meeting, the teacher and supervisor will identify a plan for the teacher 
to follow. The plan may include peer observation, workshops, or seminars that are related 
to the instructional areas on which the teacher is working, and recommendations for 
reading educational materials or viewing videotapes related to the specific domain or 
component. The teacher has a peer mentor or coach who will support and guide the 
teacher through the developmental process. 

Finally, the evaluation is conducted according to the school's established system. 
The evaluation should include the documentation of the progress made by the teacher in 
increasing the skills in the component. Both the teacher's documented evidence, 
organized in a portfolio, as well as the supervisor's observations and other data sources, 
are included to provide materials for the evaluation. According to Danielson, the 
evaluation based on the framework for teaching "takes a long view of the teacher as a 
professional, one engaged in a process of continuing improvement" (1996, p. 59). 

The steps identified by Danielson (1996) are relevant to the DLIFLC environment. 
Supervisors meet with teachers at the beginning of the evaluation year to discuss their 
performance standards. At this initial stage, the teacher and supervisor may discuss the 
teacher's goals and objectives. Throughout the year, the supervisor collects data based on 
classroom observations or other evaluation strategies. During quarterly or midpoint 
performance reviews, the teacher is encouraged to provide evidence of accomplishments 
that contribute to the teacher's achievement of the goals that have been set. Finally, a 
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summative evaluation is conducted. The summative report should document the 
contributions made by the teacher based on the performance standards. 

Summary 

The review of the literature included a discussion of the purposes of teacher 
evaluation, which were identified as being primarily for accountability or quality 
assurance and for teacher improvement or teacher development. The Personnel 
Evaluation Standards (JCSEE, 2009) were referenced as a reminder that these serve as 
guides to sound personnel evaluations. Problems in teacher evaluation were also 
summarized and two specific areas were highlighted: the lack of administrator training in 
conducting teacher evaluation and the dual roles of the evaluator as mentor or coach, in 
formative evaluation, as well as judge in summative evaluation. Empirical studies were 
included to illustrate the challenges and problems in teacher evaluation. 

Methods or strategies for evaluating teacher performance were identified in the 
literature review. Scholarly articles that described studies were included to provide 
empirical evidence of the research conducted regarding the various strategies for teacher 
evaluation. A recurring theme in the literature on teacher evaluation was the caution that 
multiple sources of data be considered when conducting teacher evaluation. The benefits 
and drawbacks of the various approaches were discussed and traditional approaches, such 
as observation and student evaluation of teaching, were described as being directive and 
characterized as being hierarchical approaches. Portfolios, action research, and self- 
evaluation were characterized as promoting self-reflection, an important aspect of 
professional growth. 
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The role of supervision and leadership related to teacher evaluation and teacher 
growth and development were also highlighted. Specifically, the roles that trust and 
performance feedback play in teacher evaluation were discussed. Trust was found to be 
an important theme throughout the studies reviewed that related to effective teacher 
evaluation. The importance of a trusting relationship between the supervisor and teacher 
was worthy of mention. Also, the importance of performance feedback to help teachers 
improve was addressed in the review. 

Two models that addressed the importance of professional growth were described: 
transformational leadership and Danielson's (1996, 2008) framework for teaching. 
Transformational leadership, which emphasizes individualized consideration, facilitates 
the link between teacher evaluation and professional development. Danielson's 
framework was described to demonstrate the steps that can be taken by teachers and their 
supervisors to build an effective teacher-evaluation process. Whereas, there is much 
scholarly literature on the topic of teacher evaluation, there are not many studies that 
specifically examine the views of both teachers and their supervisors regarding the 
teacher-evaluation strategies they believe are most beneficial to the teacher for 
professional growth and development. This study has as its objective to fill this gap in the 
literature. 
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CHAPTER III 
METHODOLOGY 
Restatement of Purpose 
The purpose of this study was to examine DLIFLC foreign-language teachers' 
and supervisors' perspectives regarding the effectiveness of teacher-evaluation strategies 
and processes they experience. The study asked DLIFLC foreign-language teachers and 
supervisors who work in the UEL about the various evaluation strategies used, such as 
classroom observation and student evaluation of teaching, and whether these contributed 
to the teachers' professional development. The foreign-language teachers and supervisors 
were specifically asked to report the extent to which these various strategies contribute 
positively to the teachers' professional development to improve teaching performance. 
The study additionally explored teachers' and supervisors' views regarding evaluation 
strategies and the role supervisors play in conducting evaluations. 

Research Setting 

The site of this study was DLIFLC, a military language school established in 1941. 
The DLIFLC is located on the Presidio of Monterey in California and is regarded as one 
of the finest schools for foreign-language instruction in the nation (DLIFLC, 2010). 
Twenty-four languages and several dialects are taught at the Institute, which employs 
over 2,000 foreign-language teachers and administrators, 98% of whom are native 
speakers of the languages they teach. Students who attend DLIFLC are members of the 
U.S. Armed Forces. Enrollment at the Institute averages approximately 3,500 students. 
Language instruction is intense, with language classes held 5 days a week, 7 hours a day. 
Language programs vary in length from 6 months to over 1 year (DLIFLC, 2010). The 
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undergraduate language program comprises eight language schools located on various 
parts of the Presidio of Monterey. The DLIFLC is a uniquely diverse educational 
institution, accredited by the Accrediting Commission for Community and Junior 
Colleges of the Western Association of Schools and Colleges. 

The present study was conducted in the UEL, an undergraduate language school 
with an average of 700 students. At the time of the study, the UEL consisted of nine 
foreign-language departments. Each department is headed by a department chair who 
supervises, on average, 20-25 foreign-language teachers and evaluates their teaching 
performance. Only those foreign-language teachers and their supervisors (i.e., department 
chairs) who participated in the evaluation process during the previous year were asked to 
participate in the study. 

Research Design 

The methodology of this mixed-methods study consisted of a self-administered, 
cross- sectional survey. Mixed-methods research is described as research that combines 
qualitative and quantitative approaches in the research methodology of a single study or 
multiphased study (Creswell, 2003; Teddlie & Tashakkori, 2008). A self-administered 
survey was determined to be the best method to collect the data for this study because this 
type of survey allows participants time for thoughtful responses. Moreover, when 
administered in group settings, self-administered surveys have a high response rate 
(Fowler, 2009). Furthermore, surveys are used to describe trends in the data in order to 
learn about a population (A. Fink, 2009) and surveys are used "to describe the attitudes, 
opinions, behaviors, or characteristics of the population" (Creswell, 2008, p. 388). 
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A survey approach to data collection was viewed as appropriate for research on 
teacher evaluation because the goal of this research was to learn about teachers' and 
supervisors' perspectives on the effectiveness of teacher evaluation at DLIFLC for the 
ultimate purpose of improving both the teacher-evaluation process and student 
achievement. Finally, a survey offered a practical and timely means of obtaining 
responses from the population who work in the UEL, one of DLIFLC 's largest 
undergraduate language schools. 

The survey included both closed- and open-ended questions. A concurrent 
triangulation strategy was employed as the data were collected on one occasion and equal 
priority was given in the interpretation phase (Creswell, 2003). The researcher included 
quantitative data to show the frequency and magnitude of trends and to collect 
information from the survey instrument that would yield statistical data (Creswell, 2003, 
2008). The qualitative data consisted of written responses to open-ended questions. These 
data were sought to clarify, elaborate, and validate the quantitative findings. Data from 
the open-ended questions were additionally included to better understand the research 
problem and to give voice to the participants. The written responses were categorized 
using content-analysis techniques, which allowed the researcher to gain direct 
information from study participants. Participants' responses, in turn, gave the researcher a 
richer understanding of the teacher-evaluation phenomenon (Hsieh & Shannon, 2005). 
Thus, the use of a survey as part of a mixed-methods study enabled both quantitative and 
qualitative data to be gathered concurrently. 
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Population and Sample 

The teacher-evaluation survey was administered to civilian foreign-language 
teachers and their supervisors (i.e., department chairs) who work in the UEL at DLIFLC. 
In fiscal year 2010, 147 foreign-language teachers were identified as having been 
evaluated. These 147 teachers and their nine supervisors thus represented the population 
for this study, as they had all participated in the DLIFLC evaluation process the previous 
year. The foreign-language teachers who worked in UEL, and who were evaluated during 
2010, taught the following languages: French, German, Hebrew, Italian, Portuguese, 
Russian, Serbian/Croatian, and Spanish. Of the 147 teachers, there were 98 females 
(67%) and 49 (33%) males. Of the nine supervisors (i.e., department chairs) who 
evaluated the teachers, there were five females (56%) and four (44%) males. 

For the purposes of this study, the population was defined as those UEL foreign- 
language teachers and their supervisors who had participated in the formal evaluation 
process the previous year. Based on the recommendations in Orcher's (2007) table of 
sample sizes, the number of teacher participants recommended to participate in the 
survey should be approximately 110. Orcher acknowledged obtaining a representative 
sample is more important than using a large sample; that is, a smaller representative 
sample is preferable to a large biased sample. According to Orcher, "If a population is 
small and accessible, it may be possible to use all members of the population as 
participants in a survey, negating the need to sample" (2007, p. 43). One advantage to 
surveying all of the foreign-language teachers in one school is that there may be more 
consistency and, therefore, less error or variation in how teacher-evaluation processes are 
implemented. 
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This study examined teachers' and supervisors' perspectives on evaluation; thus, 
it was important to ensure that participants taking the survey had a shared understanding 
of this construct. Throughout fiscal year 2010, the researcher, who is an administrator in 
the UEL, facilitated a number of discussions with supervisors and teachers regarding the 
various teacher-evaluation strategies implemented in the school. The purpose of these 
discussions was to be certain that a common understanding of the school's evaluation 
processes and strategies existed among teaching faculty and their supervisors. Fowler 
(2009) emphasized the importance of ensuring that questions "mean the same thing to all 
respondents" (p. 110). 

Human Subjects Approval 
Permission was obtained from the DLIFLC to administer the survey to foreign- 
language teaching faculty and their supervisors (i.e., department chairs) who work in the 
UEL at DLIFLC (see Appendix A). Institutional Review Board (IRB) permission was 
also granted by the University of San Francisco's IRB for the Protection of Human 
Subjects to conduct the study (see Appendix B). Data were collected in accordance with 
the rules and regulations of the University of San Francisco's IRB for Protection of 
Human Subjects. Ethical research procedures were followed throughout all phases of the 
study. The American Psychological Association's (2002) ethical principles were 
observed. These principles are (a) beneficence and nonmaleficence, (b) fidelity and 
responsibility, (c) integrity, (d) justice, and (e) respect for people's rights and dignity. 
The topic of the research — teacher evaluation — was not considered to be sensitive. In 
order to show respect for the rights of the research participants, the researcher strove to 
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protect the anonymity of the participants. Additionally, once the surveys were collected 
they were stored in a locked filing cabinet in the researcher's office. 

Instrumentation 

For this study, a survey instrument, the Teacher Evaluation Survey (TES), 
developed by McDougall (2001), was adapted to ascertain the perspectives of foreign- 
language teachers and supervisors regarding the effectiveness of the teacher-evaluation 
processes they experience. McDougall' s instrument was originally developed to collect 
data on the perceptions of public school K-12 teachers regarding various aspects of 
teacher evaluation. Permission was granted by McDougall to adapt the instrument for this 
study (see Appendix C). The McDougall TES was adapted for the two groups included in 
this study: one version for foreign-language teachers and one version for their supervisors. 

The survey was adapted to the DLIFLC environment by eliminating terminology 
specific to public schools and by replacing the term "evaluator" with "supervisor." The 
TES was also adapted so that it could be administered to DLIFLC supervisors (i.e., 
department chairs) who evaluate foreign-language teachers. TES items were modified so 
that supervisors were asked to respond to the same items as teachers. That is, the same 
questions that were asked of teachers were rewritten and posed to supervisors in order to 
compare the perspectives of both groups. The two versions of the survey can be found in 
the appendices (see Appendix D for the TES for Teachers and Appendix E for the TES 
for Supervisors). 

Prior to conducting the pilot of the survey, the researcher critically reviewed the 
survey questions in a process known as cognitive testing. Fowler (2009) suggested that, 
once written, survey items should be reviewed in a systematic and critical manner. For 
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this study the researcher followed the guidelines outlined by Fowler (2009) and 
conducted cognitive interviews by doing a think-aloud procedure with volunteers from 
DLIFLC who are knowledgeable about the content of the questions. The five think-aloud 
participants were current DLIFLC educators, and they were individuals who do not work 
in the UEL. These individuals informed the researcher about the ways they understood 
the questions and also noted their reasons for choosing specific answers. This information 
was beneficial to the researcher to ensure the questions were soliciting the information 
needed. 

The two versions of the survey, the TES for Teachers and the TES for Supervisors, 
both consisted of four categories of items that are grouped according to topic. Part I was 
divided into two sections. The first section, Part I A, Evaluation Strategies Used in the 
Teacher Evaluation Process, asked participants to identify the evaluation strategies used 
in the teacher evaluation process. Part I B, Evaluation and Professional Development, 
asked participants to indicate, using a 5-point Likert-type scale (from strongly agree to 
strongly disagree), the extent to which they agreed that the strategies listed contributed 
positively to teachers' professional development. Part I B also asked participants to share 
their written comments on the effectiveness or ineffectiveness of the evaluation strategies. 
Some of the strategies listed in these sections were classroom observation for evaluation, 
student evaluation of teachers, and performance reviews (with feedback and discussion 
with supervisor). There were a total of 11 teacher-evaluation strategies included in this 
section. Participants were given the opportunity to write an evaluation strategy not 
mentioned in the list of evaluation strategies provided. 
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Part II of both versions of the survey addressed participants' views of the 
Perceived Effectiveness of the Teacher Evaluation Process. Participants were asked to 
indicate their level of agreement or disagreement with statements using a 5-point Likert- 
type scale, from strongly agree to strongly disagree. This section of the teacher version of 
the survey contained several items that asked about teachers' perspectives regarding the 
role of the supervisor, and the corresponding section of the supervisor version addressed 
supervisors' perspectives regarding their roles as evaluators. One example item from the 
teacher version was: "My supervisor is adequately trained to evaluate my teaching 
performance." The parallel item from the supervisor version of the survey stated: "I am 
adequately trained to evaluate my teachers' performance." 

Part III of both versions of the questionnaire addressed the Perceived Objective of 
the Teacher Evaluation Process. In this section, participants were asked to express their 
level of agreement or disagreement with two items: "Improving teacher performance is 
an important objective of the evaluation process," and "Assessing quality of instruction is 
an important objective of the evaluation process." This section also asked respondents to 
indicate the main purpose of the evaluation process. Respondents were asked to select 
from among six options but could also write a purpose of their choice. 

Part IV of both versions of the survey asked respondents to provide background 
information such as their gender, age, academic rank, employment status (tenured or 
nontenured), education, foreign-language teaching experience in general, and foreign- 
language teaching experience at DLIFLC. 



94 

Pilot Study 

The TES for Teachers and the TES for Supervisors were pretested with DLIFLC 
foreign-language teachers and supervisors. Pilot- study participants were from three 
DLIFLC language schools: the Korean, Chinese, and Arabic Language Schools. The 
survey was administered to 33 foreign-language teachers and 17 supervisors. This diverse 
population of teachers and supervisors was appropriate given the similarly diverse 
teacher and supervisor population in the UEL. The instruments were parallel surveys; that 
is, they contained similar questions tailored to each population. 

The purpose of the pilot study was to assess the validity and reliability of the 
survey. The researcher also became familiar with the process of data collection and data 
analysis. Procedures were standardized throughout the administration of the surveys for 
the purpose of determining the validity and reliability of the instruments. Ethical 
procedures were followed during the conduct of the pilot study (e.g., data collection of 
the pilot study included obtaining informed consent from all participants). The results of 
the pilot study were analyzed and, based on the analysis, no changes to the two 
instruments were made. The following sections discuss the validity and reliability of the 
TES used in this study. 

Validity 

The validity of the original TES developed by McDougall (2001) was determined 
by a validity panel of 10 experts who were asked to review the TES and provide feedback 
on its face, construct, and content validity. McDougall (2001) stated that members of the 
validity panel were selected because of their background and expertise in education. 
McDougall further reported that the panel members' recommendations were incorporated 
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into the survey. Creswell (2008) confirmed that researchers generally consult with a 
panel of experts and have them identify the validity of the questions. A. Fink (2009) 
reported that content and construct validity are generally established by referring to 
theories and by asking experts if the items are representative samples of the traits and 
attitudes under investigation. 

The original TES had content validity, as McDougall's study included a panel of 
experts who examined the instrument and determined that the questions were valid. Two 
validity panels were assembled to review the adapted TES for Teachers and the TES for 
Supervisors. Members of the validity panel for the TES for Teachers were DLIFLC 
educators. These five subject-matter experts all had served as foreign-language teachers 
at DLIFLC. Four of the validity panel members worked as academic specialists involved 
in guiding and orienting new faculty members, overseeing curriculum updates, and 
working with the supervisors who evaluate the foreign-language teachers. The fifth 
member worked in the research division. The validity panel members all had experience 
with foreign-language teacher evaluation at DLIFLC, both in the role of teacher and in 
the role of evaluator. 

The validity panel that reviewed the TES for Supervisors consisted of four 
members. Three of the panel members served as deans of undergraduate language 
schools at DLIFLC. Each of these deans had experience as supervisors who had 
evaluated foreign-language teachers and department chairs. The fourth member of the 
validity panel for the review of the TES for Supervisors worked in the DLIFLC research 
division. 



96 

The members of the two validity panels independently reviewed the specific 
survey they were assigned to read for the purpose of assessing the content of the surveys. 
The researcher met with each panel member individually to discuss the survey. Thus, the 
panel members assisted the researcher in determining the content validity of the items 
contained in the instruments. The validity -panel members were primarily foreign- 
language-education specialists and included two researchers. Demographic profiles of the 
members of both validity panels are included (see Appendices F and G). The validity- 
panel members also examined the research questions, objectives of the survey, and the 
procedures that would be used to administer the survey. Based on interviews with each 
panel member, the validity reviewers of the TES reported to the researcher that the TES 
questions and procedures were appropriate for the study. 

Reliability 

The original TES, developed by McDougall (2001), was conducted as a pilot 
study. Twenty-two teachers participated in the pilot study. A. Fink (2009) suggested that 
researchers include respondents similar to the ones who will complete the survey. 
McDougall reported that the pilot survey was administered to elementary school teachers, 
a population similar to the population to whom McDougall 's final survey was 
administered. McDougall specifically asked the pilot respondents to comment on the 
clarity of the questions. Additionally, McDougall administered a retest of the pilot survey, 
which included the recommendations made by the pilot respondents, and used 
Cronbach's alpha to provide an overall assessment of the items' consistency. Pilot study 
results of the test-retest demonstrated good correlation for almost all questions. 
McDougall reported a Cronbach's alpha of .87 in the pilot study (22 cases and 23 items). 
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As previously mentioned, a pilot study also was conducted for this research 
project in the spring of 201 1. Internal-consistency reliability tests were conducted to 
determine the reliability of the instruments. Creswell (2008) reported that an instrument's 
scores are considered reliable if the scores are internally consistent, meaning that the 
scores are internally consistent across the items on the instrument. Item means replaced 
missing values in the Statistical Package for the Social Sciences (SPSS) prior to 
calculating the instruments' reliability scores. According to Creswell (2008), this 
procedure corrects for missing data. 

Two scales were created: one to measure teachers' and supervisors' perspectives 
regarding evaluation and professional development, and one to measure the effectiveness 
of the teacher-evaluation process. The Teacher-Evaluation and Professional-Development 
scale consisted of items in Part I B, Questions 2 A-K on the survey; and the Cronbach's 
alpha for this scale was a = .744. The second scale, Teacher-Evaluation-Process 
Effectiveness, consisted of items in Part II, Questions 3-19; and the Cronbach's alpha for 
this scale was a = .868. These reliability scores indicate that the survey questions have a 
high intercorrelation. The survey demonstrated appropriate reliability results as they are 
above the minimum acceptable score of .70 for measuring an instrument's internal 
consistency (Orcher, 2007). 

Once the study's data were obtained, the reliability for the two survey scales was 
assessed again. The reliability, Cronbach's alpha, of the TES scales revealed an 
acceptable level of reliability for the two scales developed for the survey: (a) Teacher 
Evaluation and Professional Development was determined to be a = .78 and (b) Teacher 
Evaluation Process Effectiveness was found to be a = .92. Cronbach's alpha was used to 
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determine each scale's internal consistency. Based on the reliability results, the TES 
items contributed to the internal consistency of the instrument. It should be noted that the 
two negatively worded items (Questions 18 and 19) were reverse coded, switching them 
from negative to positive. The reverse coding increased the Cronbach's alpha for the 
Evaluation Process Effectiveness Scale, Questions 3-19. The reverse coding was used 
throughout the data analysis of all research questions. 

Data Collection 

The following steps were taken in the data-collection process. A list of all of the 
foreign-language teachers and their supervisors who work in the UEL was obtained. The 
list accurately reflected the names of all teachers (n = 147) who had received an annual 
performance evaluation during the year prior to the study being conducted as well as the 
names of the supervisors (n = 9) who had evaluated their faculty members the prior year. 
Only foreign-language teachers and their supervisors who were currently employed in the 
UEL were surveyed. The researcher asked those 147 UEL teachers and the nine UEL 
supervisors who evaluated them to fill out the questionnaire. 

The researcher, as dean of the UEL, was able to ensure that faculty and 
supervisors understood that their participation in the survey was voluntary. To minimize 
the role of power (Creswell, 2003) and to avoid influencing participants' responses, the 
researcher left the room after explaining the purpose of the surveys to the participants. An 
assistant remained in the room to collect the completed surveys. 

The TES was administered to participants in a group setting. In order to reduce 
nonresponse, the researcher notified the participants ahead of time that they would be 
asked to participate in the survey. The survey for teachers and supervisors was 
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administered in separate meetings. The researcher included the administration of the 
teacher survey as an item on the agenda of various faculty meetings that were held across 
the school. At each faculty meeting, the researcher explained the purpose of the survey 
and informed the participants that their participation would be voluntary and thanked 
those willing to fill out the survey. Participants were told that their help was important 
(Fowler, 2009) and that they would be informed of the potential benefits that could come 
from the research. In addition, they were told that their participation was needed to 
conduct an important study on evaluation processes and that the responses they provided 
might contribute to the improvement of evaluation processes in their school and perhaps 
at DLIFLC as a whole. 

The researcher distributed the informed-consent forms (see Appendix H) and the 
Research Subjects' Bill of Rights (see Appendix I) to all participants and then left the 
premises. Respondents filled out the survey in the presence of the researcher's assistant, 
who then collected the surveys and returned them to the researcher. Fowler (2009) 
confirmed that one advantage of self-administered surveys is that cooperation rates tend 
to be high. 

A separate meeting was held for the supervisors in which they were asked to 
participate in the survey. A process similar to the one used in the administration of the 
TES for Teachers was followed. The researcher, dean of the school in which the data 
were collected, emphasized to the participants the voluntary nature of their participation 
in the study. The potential benefits of the survey were also explained to supervisors. Once 
the surveys were collected, the data were entered into SPSS and analyzed. Data for both 
surveys were anonymous and confidential. 
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Data Analysis 

The data analysis provided statistical information for each research question. 
Table 1 shows the research question, variables being studied, and the specific sections of 
the TES that measured each of the variables. 

Table 1 

Research Questions, TES Variables, and Survey Questions 



Research Questions 



Variables 



Survey Questions 



1) . What evaluation strategies do DLIFLC 
supervisors most frequently use to assess teacher 
performance? 

2) . What are DLIFLC teachers' and supervisors' 
perspectives regarding the evaluation process, and 
to what extent do the perspectives of supervisors 
and teachers differ regarding the evaluation 
strategies and their effectiveness in promoting 
professional development 

3) . What are DLIFLC teachers' perspectives 
regarding the main objective of the evaluation 
process? 

4) . What are DLIFLC supervisors' perspectives 
regarding the main objective of the evaluation 
process? 



Evaluation Strategies 
Used 



Evaluation and 

Professional 

Development 

Evaluation Process 
Effectiveness 
Role of Supervisor 

Perceived Objective of 
Evaluation Process 



Perceived Objective of 
Evaluation Process 



Part I A: A-K 

Part I B: 2 A-K 
Part II: 3-19 

Part III: 20-23 
Part III: 20-23 



The data answered the following research questions: 

1. What evaluation strategies do DLIFLC supervisors most frequently use to 
assess teacher performance? 

This research question was answered by Question 1 in Part I A of the TES for 
Teachers and the TES for Supervisors. Descriptive statistics (i.e., frequencies and 
percentages) were used to respond to this research question. 

2. What are DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, and to what extent do the perspectives of supervisors and 
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teachers differ regarding the evaluation strategies and their effectiveness in 
promoting professional development. 

This research question was answered by the items comprising Part I B, Question 2 
of the TES for Teachers and the TES for Supervisors. Means and standard deviations of 
teachers' and supervisors' perspectives regarding the 11 teacher-evaluation strategies 
were reviewed. For those evaluation strategies that showed large differences in teacher 
versus supervisor means and standard deviations, t tests were conducted to determine if 
there was a statistically significant difference. In Part I B, Question 2, participants were 
additionally asked to provide written comments to open-ended questions regarding the 
effectiveness or ineffectiveness of the evaluation strategies listed. Content analysis was 
conducted; that is, the written comments provided by participants were reviewed and 
categorized according to identified themes. 

Questions 3-19 in Part II of both surveys also contributed to answering this 
research question. The teacher and supervisor means and standard deviations for the 
effectiveness of the evaluation process were analyzed. Teachers' and supervisors' 
perspectives regarding the various evaluation processes were compared to determine if 
there was a statistically significant difference. Participants' responses to Questions 3-19 
further contributed to providing insight into the role of the supervisors as 
transformational leaders. 

To compare survey results of teachers and supervisors, two scales were created 
and results were analyzed. The first scale, Evaluation and Professional Development, 
measured the respondents' overall views of the effectiveness of the evaluation strategies 
indicated in the TES, Part I B, Question 2 A-K. Teachers' and supervisors' mean scores 
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were then compared by conducting t tests to determine if there was a statistically 
significant difference between the two groups' perspectives. A t test is described as "a 
test of statistical significance, often of the differences between two group means" (Vogt, 
2005, p. 329). 

The second scale, Evaluation Process Effectiveness, measured the teachers' and 
supervisors' perspectives regarding the various teacher-evaluation processes listed in the 
TES, Part II. Responses provided by teachers and supervisors were once again compared 
using independent t tests. Based on a review of the means and standard deviations of the 
various teacher-evaluation strategies in the TES, Part I B, Question 2 A-K, it was 
determined that those evaluation strategies that revealed rather large differences in scores 
would be further analyzed using independent sample t tests. It was then determined that 
preobservation conferences and student evaluations (Interim/End of course Student 
Questionnaires [I/ESQs]) were important to investigate, due to the differences noted in 
teachers' versus supervisors' means and standard deviations. 

As previously mentioned, in addition to the descriptive and inferential data 
analysis, content analysis of the open-ended questions was conducted to validate the 
quantitative data. As reported by Creswell (2008), open-ended responses allow the 
researcher to find information that goes beyond the closed-ended options provided to 
participants. The open-ended questions included in the TES addressed respondents' 
perspectives regarding evaluation and professional development. Part I B of the survey 
asked participants to indicate the extent to which they agreed that the teacher-evaluation 
strategies contributed to their professional development. Further, participants were given 
the opportunity to provide comments regarding the 1 1 teacher-evaluation strategies 
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included in the survey and were specifically asked to explain what makes a strategy 
effective or ineffective. 

Based on participant responses, the researcher identified primary and secondary 
themes, and responses were categorized according to each theme. A second coder 
reviewed the themes that were identified by the researcher. The process of checking for 
agreement consisted of the following steps: First, the researcher and second coder met to 
review the themes that had been identified by the researcher. Then, the second coder 
reviewed all of the open-ended responses and the corresponding themes. Lastly, the 
second coder and the researcher met to discuss the coding and themes to determine if 
there were any areas of disagreement regarding the coding. The second coder and the 
researcher concurred on the themes and the categorization of the respondents ' written 
comments. Thus, intercoder reliability was established (B. Johnson & Christiansen, 2012). 

3. What are DLIFLC teachers' perspectives regarding the main objective of the 
evaluation process? 

This research question was answered by the following items on the TES for 
Teachers: Part III, Questions 20-23. Questions 20 and 21 ask participants to indicate, 
using a 5-point Likert-type scale (from strongly agree to strongly disagree), the extent to 
which they agreed that improving teacher performance and assessing quality of 
instruction were important objectives. Questions 22 and 23 asked participants to select 
from a list of six items to complete these questions: Question 22, "The main purpose of 
my school's evaluation process is to ..." and Question 23, "My school's evaluation 
process would be more useful to me if the main purpose were to . . . ." Means and standard 
deviations regarding teachers' perspectives on the main objective of the evaluation 
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process are reported and analyzed. Cross tabulations of the teachers' responses to 
Questions 22 and 23 are included. 

4. What are DLIFLC supervisors' perspectives regarding the main objective of 
the evaluation process? 

This research question is similar to the previous question; however, it examined 
the supervisors' perspectives regarding the main objective of the evaluation process. The 
question was answered by the following items on the TES for Supervisors: Part III, 
Questions 20-23. Questions 20 and 21 asked participants to indicate, using a 5-point 
Likert-type scale (from strongly agree to strongly disagree), the extent to which they 
agreed that improving teacher performance and assessing quality of instruction were 
important objectives. Questions 22 and 23 asked participants to select from a list of six 
items to complete these statements: Question 22, "The main purpose of my school's 
evaluation process is to..." and Question 23, "My school's evaluation process would be 
more useful to me if the main purpose were to. . . ." Means and standard deviations 
regarding supervisors' perspectives on the main objective of the evaluation process are 
reported and analyzed. Cross tabulations of the supervisors' responses to Questions 22 
and 23 are included. 

In addition to the analysis of information regarding the research questions, 
demographic information on teachers and their supervisors (i.e., department chairs) was 
also analyzed. The demographic information was analyzed using descriptive statistics, 
and results are reported as frequency distributions and percentages. Tables of descriptive 
statistics (e.g., means and standard deviations) are included. 
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Background of the Researcher 
This section describes the academic and professional background of the 
researcher. The researcher is a graduate of the University of Texas at Austin (B.A. in 
Spanish Language and Literature). The researcher further studied at the University of San 
Francisco (M.A. in Multicultural Education) and the Monterey Institute of International 
Studies (M. A. in Teaching Foreign Languages, Spanish). 

The researcher serves as academic dean of one of DLIFLC's undergraduate 
schools, the UEL. At the time of the study there were approximately 200 civilian and 20 
military foreign-language teachers in the school and approximately 700 students enrolled 
in the eight language programs taught in UEL. Prior to serving as dean of UEL, the 
researcher served as department chair for the Spanish and Serbian/Croatian departments 
at the DLIFLC. The researcher began a career at DLIFLC as a Spanish-language teacher 
and later worked as an academic specialist, responsible for providing curriculum and 
instructional support for the various romance languages taught at DLIFLC. Prior to 
working at the DLIFLC, the researcher worked at the Defense Language Institute English 
Language Center, teaching ESL to foreign military personnel. 

The researcher has over 30 years of experience in the foreign-language education 
field and over 20 years of experience conducting teacher evaluations. Throughout the 
years as a supervisor responsible for evaluating foreign-language teachers and foreign- 
language supervisors, the researcher has observed the tension that teachers and 
supervisors experience as evaluatees and evaluators. As the researcher noted, both in the 
research literature and in personal experience, much time and energy is spent on teacher 
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evaluation, but unfortunately, this process rarely seems to contribute to teachers' 
professional growth and development. 

Summary 

DLIFLC foreign-language teachers are evaluated for the purposes of quality 
assurance and for professional development according to DA Regulation 690-400 (1998). 
This regulation highlights the role that professional development plays in the evaluation 
process. Teachers' and supervisors' perspectives are an important part of the teacher- 
evaluation process as they are the primary agents involved in this process. This study 
examined the views of foreign-language teachers and supervisors regarding teacher- 
evaluation strategies and processes that contribute to teachers' professional development. 
Foreign-language teachers and supervisors were also asked to share their views regarding 
the objectives and purpose of the evaluation process. In addition, the study explored the 
role of the supervisor in teacher evaluation, seen through the model of transformational 
leadership. 

The scope of this study was to gain insight into the perspectives of foreign- 
language teachers and supervisors employed at DLIFLC 's UEL regarding the teacher- 
evaluation strategies and processes they experience. A mixed-methods research approach 
was implemented. The mixed-methods research design thus included quantitative 
(closed-ended responses to the survey) and qualitative (open-ended responses to the 
survey) data. 

The next chapter contains information obtained from respondents, including 
demographic data, displayed in tables. Each research question is addressed and the 
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corresponding data are analyzed. The chapter concludes with a summary of the 
quantitative and qualitative responses. 
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CHAPTER IV 
FINDINGS 
Introduction 

This study examined the perspectives of foreign-language teachers and 
supervisors regarding the effectiveness of various teacher-evaluation strategies and 
processes. In addition, the study explored teachers' and supervisors' views regarding 
evaluation strategies and the role supervisors play in conducting the evaluations. The 134 
foreign-language teachers and nine supervisors who participated in the study were 
employed at DLIFLC's UEL. This chapter describes the quantitative and qualitative 
findings based on the teachers' and supervisors' responses to a TES. 

The following research questions were examined in this study: 

1. What evaluation strategies do DLIFLC supervisors most frequently use to 
assess teacher performance? 

2. What are DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, and to what extent do the perspectives of supervisors and 
teachers differ regarding the evaluation strategies and their effectiveness in 
promoting professional development? 

3. What are DLIFLC teachers' perspectives regarding the main objective of the 
evaluation process? 

4. What are DLIFLC supervisors' perspectives regarding the main objective of 
the evaluation process? 

The results of the study are presented as follows: First, a demographic description 
of the study's participants is provided. Second, answers to each of the research questions 
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are presented and data and statistical results are displayed in table format. As the TES 
contained both closed and open-ended questions, responses in narrative form are also 
provided. Third, a summary of the chapter is included. 

Participant Profile 

The supervisors' and teachers' demographic profiles presented in Table 2 include 
gender, age, tenure status, academic rank, and degrees earned. The TES for Teachers and 
Supervisors was administered to the participants from February to April 201 1. All nine 
(100%) of the supervisors completed the TES for Supervisors and 134 (91%) of 147 
teachers completed the TES for Teachers. The demographic profile shows that, in gender, 
the supervisor group is almost evenly divided, whereas, the ratio of female-to-male 
teacher respondents was more than 3:1. Regarding tenure status, it is interesting to note in 
Table 2 that five supervisors (56%) are tenured; whereas, four (44%) are nontenured. Of 
teachers, 129 provided their tenure status; 78 (61%) of these teachers reported that they 
are nontenured. 

Table 3 lists foreign-language teaching experience for both groups of respondents. 
It is interesting to note that two-thirds of the teachers have more than 10 years of general 
foreign-language teaching experience, and over three fourths of supervisors have the 
corresponding amount of general foreign-language teaching experience. Seven (78%) of 
the supervisors have more than 10 years of general foreign-language teaching experience 
and only two (22%) have 10 years or less experience teaching foreign languages. 

Upon examining teaching experience specifically at DLIFLC, the data in Table 3 
show that 99 (74%) of the teachers have taught at DLIFLC for less than 10 years. A slight 



110 



majority of the supervisors, five (56%), have more than 10 years of experience teaching 
at DLIFLC and four (44%) have 5 years or less experience. 

Table 2 

Demographic Indicators for Supervisors and Teachers 



Supervisors 
(n = 9) 



Teachers 
(«= 134) 



Demographic category 



/ 



/ 



Gender 
Female 
Male 

Age 

21-30 years 

31—40 years 

41-50 years 

51-60 years 

Over 60 years 
Tenure Status 

Tenured 

Nontenured 
Academic Rank 

Professor 

Associate professor 
Assistant professor 
Senior instructor 
Instructor 
Degree 
BA 
MA 

Ph.D./Ed.D. 



5 
4 

1 

2 
2 
3 
1 

5 
4 

2 
1 
6 
0 
0 

0 
6 

3 



55.6 
44.5 

11.1 
22.2 
22.2 
33.3 
11.1 

55.6 
44.4 

22.2 
11.1 
66.7 

0 

0 

0 
66.7 

33.3 



100 
30 

2 
34 
41 
32 
18 

51 
78 

1 

21 
107 
1 

2 

22 
92 
16 



74.6 
22.4 

1.5 
25.4 
30.6 
23.9 
13.4 

38.1 
58.2 

0.7 
15.7 
79.9 
0.7 
1.5 

16.4 
68.7 
11.9 



Note: Some participants did not respond to demographic items; therefore, not all frequencies total 143 and 
percentages in these cases were less than 100. 
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Table 3 

Demographic Indicators (Teaching) for Supervisors and Teachers 



Supervisors Teachers 
(n = 9) (n=134) 



Demographic category 


f 


% 


/ 


% 


Foreign language 










teaching experience 










0-5 years 


1 


11.1 


16 


11.9 


6-10 years 


1 


11.1 


29 


21.6 


11-15 years 


2 


22.2 


28 


20.9 


16-20 years 


2 


22.2 


24 


17.9 


21-25 years 


1 


11.1 


11 


8.2 


26-30 years 


2 


22.2 


13 


9.7 


Over 30 years 


0 


0 


12 


9.0 


DLIFLC teaching 










experience 










0-5 years 


4 


44.4 


81 


60.4 


6-10 years 


0 


0 


18 


13.4 


11-15 years 


2 


22.2 


10 


7.5 


16-20 years 


1 


11.1 


9 


6.7 


21-25 years 


0 


0 


6 


4.5 


26-30 years 


2 


22.2 


4 


3.0 


Over 30 years 


0 


0 


3 


2.2 



Note: Some participants did not respond to demographic items; therefore, not all frequencies total 143 and 
percentages in these cases were less than 100; DLIFLC = Defense Language Institute Foreign Language 
Center. 



Table 4 presents the number of years of experience supervisors (i.e., department 
chairs) possess in the capacity of supervisor. Supervisors are responsible for evaluating 
the performance of their faculty members. As shown in Table 4, the majority of the UEL 
supervisors, six (67%), have 5 years or less experience serving as department chairs. The 
remaining three respondents (33%) represent a range in experience of 6-25 years of 
supervisory experience. 
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Table 4 

Supervisors ' Experience 



Supervisors 
(n = 9) 



Supervisory experience 


/ 


% 


0-5 years 






6-10 years 


6 


66.7 


11-15 years 


1 


11.1 


16-20 years 


1 


11.1 


21-25 years 


0 


0 


26-30 years 


1 


11.1 


Over 30 years 


0 


0 




0 


0 



The large number of supervisors with less than 5 years of supervisory experience 
is due to the large growth in hiring that has taken place in the 3 to 5 years prior to the 
study. The six supervisors (67%) with less than 5 years of supervisory experience at 
DLIFLC were hired as foreign-language teachers and have been promoted to serve as 
department chairs. The demographic data were included solely to provide the reader with 
an understanding of the background of the participants. The demographic data served the 
purpose of describing the participants. 

Results for Research Question 1 

What evaluation strategies do DLIFLC supervisors most frequently use to assess 
teacher performance? 

Use of Teacher Evaluation Strategies: Rank Ordered 

This research question was addressed by Part I of the TES for Teachers and 
Supervisors, Items A-K, to determine the teacher-evaluation strategies that supervisors 
use and teachers experience in their evaluations. Table 5 includes the frequency and 
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percentage of use or experience regarding the 1 1 teacher-evaluation strategies included in 
the survey. The data shown in the table include both supervisor and teacher totals. The 
evaluation strategies are listed in the table in order of most used to least used or 
experienced. 

Table 5 

Supervisors ' and Teachers ' Perspectives on Use of Teacher Evaluation Strategies 



Supervisors and teachers 
(n = 143) 



Teacher Evaluation Strategy 


/ 


% 


Classroom observation 


143 


100.0 


Student evaluations (I/ESQs) 


141 


98.6 


Postobservation conference 


135 


94.4 


Performance reviews 


132 


92.3 


Student feedback from sensing session 


117 


81.8 


Written observation report 


112 


78.3 


Students' performance on standardized tests 


83 


58.0 


Preobservation conference 


77 


53.8 


Self-evaluation 


70 


49.0 


Portfolios 


67 


46.9 


Action research 


39 


27.3 



Note. I/ESQs = Interim/End of course Student Questionnaires. 



The data in Table 5 reveal that, of the 1 1 strategies listed, all of the supervisors 
and teachers who responded were in agreement that the first evaluation strategy listed in 
the table — classroom observation — was used or experienced as part of the evaluation 
process. All respondents indicated that this strategy was used in the evaluation process 
100% of the time. 

The second most used teacher-evaluation strategy was student evaluations, 
specifically the I/ESQs. These student feedback questionnaires are administered twice 
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during the course of a language program. Of respondents, 141 (99%) reported use of this 
teacher-evaluation strategy. The third most used teacher-evaluation strategy was the 
postobservation conference, reported by 135 (94%) of respondents reporting its use. The 
fourth and fifth teacher-evaluation strategies listed by frequency were performance 
reviews, 132 (92%), and student feedback from sensing sessions, 117 (82%). 

Teacher-Evaluation Strategies: Least Used 

The two least used teacher-evaluation strategies were portfolios, 67 (47%), and 
action research, 39 (27%). It is important to note that, based on the foreign-language 
teachers' and supervisors' performance standards, action research is required of all 
DLIFLC teaching faculty (including department chairs) at the rank of Associate Professor 
or Professor. Only 22 (16%) of the UEL teachers who participated in the survey are 
Associate Professors or Professors, and only three (33%) of the supervisors hold these 
ranks. Teachers who hold other academic ranks are encouraged to conduct action 
research; however, they are not required to do so. 

It should additionally be noted that portfolios are not a required part of the 
DLIFLC teacher-evaluation process; however, they were included in the survey's list of 
teacher-evaluation strategies because of the strong recommendations found in the 
literature regarding the effectiveness of portfolios as a tool to promote reflection and thus 
professional development. Despite the fact that teachers and supervisors reported that 
portfolios were the least used strategy, two positive comments were provided. One 
teacher noted, "Portfolios provide more substantial evidence as to the skill of the teacher." 
One supervisor who agreed that portfolios promote teachers' professional development 
wrote, "Shows the way they present teaching materials visually." 



115 

Despite these positive views of portfolios, 3 of the 24 teachers who provided 
comments indicated that due to the high number of teaching hours required of DLIFLC 
faculty, they would have difficulty finding time to prepare portfolios for teacher 
evaluation. One teacher expressed a view that was reflected in the literature on portfolios 
by writing, "Time consuming, not sure that it is a positive contribution at all." 

Results for Research Question 2 

What are DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, and to what extent do the perspectives of supervisors and teachers 
differ regarding the evaluation strategies and their effectiveness in promoting 
professional development? 

This research question was addressed by survey questions in Part I B, items 2A- 
2K, and Part II, Items 3-19, of both versions of the TES; that is, the version for teachers 
and the version for supervisors. The responses to this question include both quantitative 
and qualitative data. The qualitative data (teachers' and supervisors' written comments) 
are taken directly from the survey used in the study. 

The quantitative data are presented initially, followed by the qualitative data. 
Qualitative data for the following teacher-evaluation strategies are presented and 
discussed: postobservation conferences, performance reviews, and classroom 
observations. Respondents' written comments for these three teacher-evaluation 
strategies were included because they were identified by the foreign-language teachers 
and supervisors as being the most effective in promoting professional development. 
Supervisors' written comments are initially presented followed by the foreign-language 
teachers' written comments regarding the effectiveness of these strategies. 
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Qualitative data on preobservation conferences and SET are also presented and 
discussed because a statistically significant difference was found when comparing the 
perspectives of teachers and supervisors regarding the effectiveness of these two 
evaluation strategies. The researcher believed that the inclusion of respondents' written 
comments would aid in achieving a better understanding of their views. 

With respect to SET, it is important to mention that the scholarly literature 
regarding this specific teacher-evaluation strategy is quite vast. The multitude of studies 
on SET highlights the controversy that often arises when using this teacher-evaluation 
strategy. Thus, the researcher believed it was important to provide as much qualitative 
data as possible regarding this strategy in particular in order to compare the findings of 
this study to the findings in previous studies. Supervisors' written comments precede the 
teachers' written comments. Whereas, the aforementioned strategies are described both 
quantitatively and qualitatively, the remaining six teacher evaluation strategies are 
described quantitatively. 

Table 6 contains data regarding the effectiveness of the various teacher-evaluation 
strategies reported by supervisors, ranked from most to least effective based on an 
analysis of means and standard deviations. 

Effectiveness of Teacher Evaluation Strategies 

The data in Table 6 show that supervisors found postobservation conferences 
(M = 4.67, SD = .50), performance reviews (M = 4.56, SD = .53), preobservation 
conferences (M = 4.44, SD = .53), and classroom observations (M = 4.44, SD = .73) most 
effective in promoting teachers' professional development. The teacher-evaluation 
strategies that were reported by supervisors to be the least effective in promoting teachers' 
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professional development were the use of students' performance on standardized tests 
(Af = 3.67, SD = 1.00), portfolios (Af = 3.56, SD = 1.01), and action research (Af = 3.33, 
SD = .71). Supervisors were consistent in their responses regarding the effectiveness of 
the use of student evaluations (Af = 4.1 1, SD = .33) as part of the teacher-evaluation 
process in particular, as evidenced by the low standard deviation for this item. 

Table 6 

Supervisors ' Perspectives of Effectiveness of Teacher Evaluation Strategies 



Supervisors 
(n = 9) 



Rank order 


Teacher evaluation strategy 


M 


SD 


1. 


Postobservation conference 


4.67 


0.50 


2. 


Performance reviews 


4.56 


0.53 


3. 


Preobservation conference 


4.44 


0.53 


4. 


Classroom observation 


4.44 


0.73 


5. 


Written observation report 


4.22 


0.67 


6. 


Student evaluations (I/ESQs) 


4.11 


0.33 


7. 


Self-evaluation 


3.89 


1.05 


8. 


Student feedback from sensing sessions 


3.78 


0.97 


9. 


Students' performance on standardized tests (DLPTs) 


3.67 


1.00 


10. 


Portfolios 


3.56 


1.01 


11. 


Action research 


3.33 


0.71 



Note. I/ESQ = Interim/End of course Student Questionnaires; DLPT = Defense Language Proficiency Test. 



A further review of the means and standard deviations of each teacher-evaluation 
strategy revealed that supervisors are in agreement that postobservation conferences, 
performance reviews, and preobservation conferences are the most effective teacher- 
evaluation strategies. The large standard deviations found for the use of self-evaluation 
(Af = 3.89, SD = 1.05), portfolios (Af = 3.56, SD = 1.01), and students' performance on 
standardized tests (Af = 3.67, SD = 1.00), on the other hand, indicate a greater disparity of 
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opinion regarding these strategies' effectiveness. The supervisors' perspectives on 
effective teacher-evaluation strategies are contrasted with the teachers' perspectives 
regarding the effectiveness of the strategies. 

The data in Table 7 reflect the teachers' perspectives regarding the effectiveness 
of the various evaluation strategies. These data show that teachers reported that the most 
effective teacher-evaluation strategies were postobservation conferences (M = 4.22, 
SD = .98), performance reviews (M = 3.97, SD = .96), classroom observations (M = 3.96, 
SD = 1.08), and written observation reports (M = 3.90, SD = .95). 

Table 7 also illustrates that teacher-evaluation strategies that were reported by 
teachers to be the least effective in contributing to teachers' professional development 
were preobservation conferences (M = 3.52, SD = 1.17), action research (M = 3.46, 
SD = .86), and portfolios (M = 3.42, SD = 1.07). 

Table 7 

Teachers ' Perspectives of Effectiveness of Teacher Evaluation Strategies 



Teachers 
(n= 134) 



Rank order 


Teacher evaluation strategy 


M 


SD 


1. 


Postobservation conference 


4.22 


.98 


2. 


Performance reviews 


3.97 


.96 


3. 


Classroom observation 


3.96 


1.08 


4. 


Written observation report 


3.90 


.95 


5. 


Student feedback from sensing sessions 


3.81 


.95 


6. 


Self -evaluation 


3.75 


1.07 


7. 


Students' performance on standardized tests (DLPTs) 


3.57 


1.04 


8. 


Student evaluations (1/ESQs) 


3.55 


1.19 


9. 


Preobservation conference 


3.52 


1.17 


10. 


Action research 


3.46 


.86 


11. 


Portfolios 


3.42 


1.07 



Note. DLPT = Defense Language Proficiency Test; 1/ESQ = Interim/End of course Student Questionnaires. 
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Effectiveness of Teacher Evaluation Strategies: Quantitative and Qualitative Data 
This section includes quantitative and qualitative data regarding the most effective 
teacher-evaluation strategies. Table 8 shows the ranking of the strategies identified by 
teachers and supervisors as being the most effective in promoting teachers' professional 
development. Postobservation conferences, performance reviews, and classroom 
observation were considered to be the most effective teacher-evaluation strategies by 
supervisors and teachers. As seen in Table 8, whereas, postobservation conferences and 
performance reviews were identified by both teachers and supervisors as the top two 
teacher-evaluation strategies for promoting teachers' professional development, 
classroom observations were found to be the third most effective evaluation strategy for 
teachers (M = 3.96, SD = 1.08); but the fourth most effective for supervisors (M = 4.44, 
SD = .73). 

Table 8 

Supervisor and Teacher Ranking of Most Effective Evaluation Strategies 



Evaluation strategy 


Supervisors and teachers 
(n = 143)) 


Supervisor ranking 


Teacher ranking 


Postobservation conferences 


1 


1 


Performance reviews 


2 


2 


Classroom observation 


4 


3 



To provide a richer description and understanding of the participants' experiences 
with teacher evaluation, respondents' written comments regarding postobservation 
conferences are provided in the next section. Supervisors' comments are presented 
initially and then teachers' comments are included. By incorporating samples of 
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respondents' written discourse, further understanding and meaning can be gained 
(Lichtman, 2006). 
Postobservation Conferences 

Postobservation conferences were, according to most supervisors and teachers, 
the teacher-evaluation strategy that most promotes teachers' professional development. 
The open-ended responses regarding postobservation conferences confirmed the 
quantitative data obtained; that is, the comments were overwhelmingly positive. 

All nine supervisors (100%) reported that postobservation conferences are 
effective in promoting teachers' professional development, with three (33%) supervisors 
expressing agreement and six (67%) strong agreement. The mean for supervisors' 
perspectives on the effectiveness of this evaluation strategy was 4.67 (SD = .50). Six of 
the nine (67%) supervisors provided comments regarding postobservation conferences. 
Of the six supervisors who wrote comments, four supervisors (67%) strongly agreed and 
two (33%) agreed that this teacher-evaluation strategy was effective in contributing to 
teachers' professional development. Two themes were identified for supervisors' 
postobservation conferences: Self-Reflection and Feedback. Sample comments that 
exemplify respondents' perspectives are provided. 

Supervisors ' open-ended responses: Postobservation conferences. 

Supervisors' Comments — Theme: Self-Reflection 

• It leads to self-reflection, potentially resulting in self-directed growth. 

• Teacher shows the ability to evaluate his own lesson, to show the willingness 
to grow and improve. 

• Used for suggestions and reflections. 
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Supervisors' Comments — Theme: Feedback 

• Feedback to help teacher improve. 

To have opportunities to discuss feedback and give own perspective on lesson. 

Teachers ' open-ended responses: Postobservation conferences. 

The teachers' mean for postobservation conferences was previously reported 
(M = 4.22, SD = .98). The analysis of teacher responses in terms of agreement with 
Question 2 C regarding the effectiveness of postobservation conferences was as follows: 
1 16 teachers (87%) strongly agreed or agreed that this evaluation strategy contributed to 
their professional development, nine (7%) strongly disagreed or disagreed and nine (7%) 
were undecided. Thirty-seven teachers provided comments and, of this number, 32 (87%) 
strongly agreed or agreed with the item. Two of the 37 (5%) respondents strongly 
disagreed or disagreed and three (8%) were undecided. The teachers' comments were 
categorized into two primary themes: Feedback and Benefits. 

Teachers ' open-ended responses: Postobservation conferences. 

Teachers' Comments — Theme: Feedback 

• Obviously feedback to improve or continue doing a good job. 

• It helps to get feedback orally and in writing. 

• It is important to have another perspective of what's happening in the 
classroom as well as feedback with suggestions to improve. 

• It's always useful to hear someone else's professional opinion about your 
work. 

• Too critical and subjective. 
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Teachers' Comments — Theme: Benefits 

• Can be a great learning experience. 

• Provides good communication. 

• Some tips can be helpful! 

Feedback was a common theme in the supervisors' and teachers' comments. 
Respondents' comments were mostly positive regarding the importance of feedback 
following the postobservation conferences and the benefits of the conferences, although 
one teacher noted that postobservation conferences were critical and subjective. 
Performance reviews are discussed in the next section, which contains sample written 
comments provided by respondents regarding the effectiveness of performance reviews. 
Performance Reviews 

Teachers and supervisors further agreed that the second most effective teacher- 
evaluation strategy was performance reviews. In UEL, supervisors and teachers meet at 
least four times during the evaluation year to discuss the teachers' performance as part of 
the teachers' performance reviews. Performance reviews were the evaluation strategy that 
had the second highest mean as reported by teachers (M = 3.97, SD = .96) and 
supervisors (M = 4.56, SD = .53). All nine supervisors (100%) reported that this teacher- 
evaluation strategy contributed to teachers' professional development, with four 
supervisors (45%) expressing agreement and five (55%) strong agreement. Open-ended 
responses were provided by five (56%) of the nine supervisors, and all who provided 
comments strongly agreed or agreed with the item. Supervisors' written comments are 
provided, followed by teachers' written comments. The prominent theme among 
supervisory comments addressed the Teachers' Perceptions of Self. 
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Supervisors ' open-ended responses: Performance reviews. 
Supervisors' comments — Theme: Teachers' Perceptions of Self 

• Let's [sic] me see how the teacher perceives himself. 

• Although sometimes difficult, it is important to show how teachers' 
performance is perceived. 

Teachers ' open-ended responses: Performance reviews. 

Thirty-one of the 134 (23%) teachers who responded to Question 2 G regarding 
the effectiveness of performance reviews provided comments. The extent of agreement 
among teachers was 102 (76%) strongly agreed or agreed, 11 (8%) strongly disagreed or 
disagreed, and 21 (16%) were undecided. Of the 31 respondents who commented, 23 
(74%) strongly agreed or agreed and 3 (10%) strongly disagreed or disagreed. Five (16%) 
respondents were undecided regarding their opinion of this evaluation strategy's 
effectiveness. Two themes were identified for teachers' comments on performance 
reviews: Feedback and the Supervisor's Role. 

Teachers' Comments — Theme: Feedback 

• Performance reviews need to happen at every job in order to have qualified 
personnel. 

• I like these reviews no matter how positive or critical it [sic] can be; I'd like to 
grow and develop in my profession. 

• Helps me to understand any deficiencies I have and to reinforce my positive 
features and methods in the teaching process. 

• Supervisor's feedback can be very subjective and does not always reflect the 
reality. 
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• Unfortunately, my performance is being evaluated only through the prism of 
I/ESQs. One student's comment/opinion is the theme of our discussion for 
years. 

Teachers' Comments — Theme: The Supervisor's Role 

• My supervisor always finds something positive to say about my performance. 
That puts me in a good mood, after discussions with her I feel needed and 
appreciated as a teacher. Even negative feedback is presented in a positive 
way. 

• If it is a knowledgeable and fair supervisor it's ok, but if not, it means 
absolutely nothing. 

• Are never a two-way street. 

• Purpose — to find weak points. 

• Biased. 

Supervisory comments were uniformly positive regarding the effectiveness of 
performance reviews, whereas, teachers expressed both positive and negative comments 
regarding performance reviews. Teachers' comments focused on the themes of Feedback 
and the Supervisor's Role. The next section presents respondents' comments regarding 
the effectiveness of classroom observations. 
Classroom Observations 

Whereas supervisors and teachers agreed that classroom observations are 
effective in promoting teachers professional development, the supervisors (M = 4.44, 
SD = .73) were more in agreement with this item than teachers (M = 3.96, SD = 1.08). 
Eight supervisors (89%) strongly agreed or agreed that classroom observations are 



125 

effective, and one supervisor was undecided regarding the effectiveness of this teacher- 
evaluation strategy. The majority of the teachers (76%) also strongly agreed or agreed 
that this was an effective teacher-evaluation strategy; however, 10% of the teachers 
strongly disagreed or disagreed. 

Open-ended responses were provided by six (67%) of the nine supervisors, and 
five (83%) of those who provided comments strongly agreed or agreed with the item. 
One supervisor who provided comments was undecided regarding the effectiveness of 
this evaluation strategy. 

Supervisors ' open-ended responses: Classroom observations. 

Supervisors' Comments — Theme: The Formative and Summative Nature of 
Evaluations 

• Not only for summative purposes, but it is a key source of information to 
assess performance and introduce change. 

• When it's done for the purpose of evaluation, the feedback is mainly directive. 
It mostly raises awareness but not necessarily leads to development. 

• Let's [sic] me see the teacher implementing his knowledge and skills in 
practice. Potential for professional development. 

• Hands-on knowledge of the person, knowing that it is sometimes artificial (e.g. 
Teachers make special effort to "please" observer — use [target language], 
etc.). 

Teachers ' open-ended responses: Classroom observations. 
Forty-two teachers provided comments regarding the effectiveness of classroom 
observations and their effectiveness in promoting teachers' professional development. 
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Twenty-nine (69%) of those teachers who commented regarding this evaluation strategy 
responded that they strongly agreed or agreed with this item, and eight (19%) of those 
teachers who commented on this evaluation strategy strongly disagreed or disagreed. 
Three themes were identified for this evaluation strategy: Feedback, Effectiveness, and 
Negative Impact. 

Teachers' comments — Theme: Feedback 

• Second opinion (view from outside) is always valuable. 

• My strong and weak points can be clarified. 

• Helps to reveal weak areas. 

• It's rather effective for it improves the quality of teaching and opens up new 
teaching methods and approaches. 

• It is always important to know if we are doing well. The opinion of colleagues 
or supervisors is very helpful. 

Teachers' Comments — Theme: Effectiveness 

• Classroom observation will affect my performance either negatively or 
positively on a given day in a given class; however, I don't believe it affects 
my professional growth. Experience and reflection do make a difference in my 
teaching practice. 

• Provides professional growth. 

• It doesn't help my professional development in any way. It just helps my 
supervisor to evaluate my existing level of professionalism. 

• Classroom observation can be very effective, but it was non-effective so far. 
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• Teachers teach to ICC [Instructor Certification Course] expectations when 
being observed 

• It's a way my supervisor can see the way I prepare However, I'd rather 

instructors are not forewarned when they are going to be observed so that 
supervisors can really see their performance. 

Teachers' Comments — Theme: Negative Impact 

• Very intimidating part of the job. 
Disturbs students and teachers. 

• They are useful but unannounced classroom observations make me and my 
students nervous. 

• I am usually extremely nervous when being observed; therefore, I feel that my 
performance is never at its best. 

The majority of the supervisors' comments were positive; whereas, teachers' 
comments included both positive and negative comments. Some teachers commented on 
the supervisor's role in providing feedback and others expressed concerns regarding the 
negative impact of unannounced observations. 

Evaluation and Professional-Development Scale 

Teachers and supervisors reported that the top three teacher evaluation strategies 
were postobservation conferences, performance reviews, and classroom observations. 
The open-ended comments regarding the top three strategies provided additional insight 
into the views of the participants. To further respond to Research Question 2, the 
Evaluation and Professional Development Scale was developed. An independent samples 
t test was conducted to compare supervisors' and teachers' perspectives regarding 
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teacher-evaluation strategies. The quantitative data will be presented initially followed by 
the open-ended comments provided by respondents on the surveys. 

As indicated in Table 9, supervisors' responses reveal a higher mean than teachers 
for the Evaluation and Professional Development Scale. Respondents were asked to 
indicate to what extent a given strategy contributed positively to teachers' professional 
development. This scale has a mean of 4.06 for supervisors, compared to 3.73 for 
teachers. 

Table 9 

Means and Standard Deviations for Evaluation and Professional Development Scale 



Scale 


Supervisor/teacher 


N 


M 


SD 


Evaluation and Professional 


Supervisor 


9 


4.06 


.440 


Development 








.578 


Teacher 


134 


3.73 



To determine if there was a statistically significant difference between responses 
provided by supervisors and teachers for the scale, an independent samples t test was 
conducted. Due to the unbalanced size of the two sample groups, nine supervisors and 
134 teachers, caution must be applied in analyzing the results. Upon examining the 
results of the independent t tests for the Scale for Evaluation and Professional 
Development (Question 2 A-K of the TES), the null hypothesis was retained; that is, no 
statistically significant difference existed between supervisors' and teachers' responses to 
questions related to Evaluation and Professional Development (t hi = 1.64, p = .103). 

To further determine if there were differences in the perspectives held by teachers 
and supervisors regarding the 1 1 teacher-evaluation strategies listed in the TES, the 
means and standard deviations for these two groups were also reviewed. Based on this 
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review, two strategies that showed large differences in means and standard deviations 
were then selected: preobservation conferences and I/ESQs. In the following section both 
quantitative and qualitative data are presented to illustrate the differences in perspectives 
between supervisors and foreign-language teachers. Preobservation conferences are first 
introduced, followed by student evaluation of teaching. 
Preobservation Conferences 

The means and standard deviations for preobservation conferences between 
supervisors (M = 4.44, SD = .53) and teachers (M = 3.52, SD =1.17) were compared. The 
data in Table 10 illustrate that supervisors are convinced of the merits of the 
preobservation conferences with regard to the positive contribution they believe they 
make to the teachers' professional development. 

Table 10 data further reveal that 100% of the supervisors strongly agreed or 
agreed that this teacher-evaluation strategy is effective; whereas, only 48% of the 
teachers found this teacher-evaluation strategy effective. A relatively large number (37%) 
of the teachers who responded were undecided regarding this teacher-evaluation 
strategy's effectiveness. 



Crosstabulation of Supervisors ' and Teachers ' Perspectives on the Effectiveness of 
Preobservation Conferences 



Table 10 



Preobservation conference 



Strongly disagree Disagree 

f % f % 



Undecided 



/ 



% 



/ 



Agree 



Strongly agree Total 

/ % 



Supervisor 0 0 00005 56 4 44 9 

Teacher 10 8 10 8 49 37 30 22 35 26 134 
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Additionally, an independent samples t test was conducted to determine if there 
was a statistically significant difference between supervisors' and teachers' perspectives 
regarding preobservation conferences. For preobservation conferences, p = .00 and 
a = .05; therefore, the null hypothesis was rejected and the alternative hypothesis was 
accepted. The results suggest that there is a statistically significant difference between the 
perspectives of teachers and supervisors regarding the preobservation conference 
(tu\ = 2.33, p = .00). Cohen's d was found to be .80, which is considered to be a rather 
large effect size. To gain further insight into the perspectives of supervisors and teachers 
regarding preobservation conferences, comments on the open-ended questions were 
examined. Open-ended responses regarding the degree of effectiveness of preobservation 
conferences are provided in the next section. Supervisory comments are presented 
initially, followed by teachers' comments. 

Supervisors ' open-ended responses: Preobservation conferences. 

Supervisors believed more strongly than teachers that preobservation conferences 
are effective in promoting teachers' professional development. All of the supervisors 
strongly agreed or agreed with this item. Five supervisors agreed (56%) and four (44%) 
strongly agreed that preobservation conferences are effective. One theme was identified 
for supervisors' comments on preobservation conferences: Benefits. Sample comments 
that exemplify supervisor perspectives are provided. 

Supervisors' Comments — Theme: Benefits 

The preobservation conference is usually a lead-in to the focus of the 
following classroom visit. Preobservation meetings are useful, to agree on 
area to be observed and its relation to the teacher's performance. 
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• It assists the teacher with developing lesson planning techniques and 
structuring the lesson. 

• Give's [sic] idea on teacher's planning. 

Teachers ' open-ended responses: Preobservation conferences. 

The extent of agreement among teachers who responded to this question was 65 
(49%) strongly agreed or agreed, 49 (37%) strongly disagreed or disagreed, and 20 (15%) 
were undecided. Thirty-four (25%) of the 134 teachers who responded to this question 
provided comments. Of the 34 respondents who provided comments, 19 (56%) strongly 
agreed or agreed with this item and nine (27%) strongly disagreed or disagreed. Six 
(18%) respondents were undecided regarding their opinion of this evaluation strategy's 
effectiveness. Teachers' open-ended comments were categorized into three primary 
themes: Degree of Helpfulness, Supervisors' Role, and Nonconduct of Preobservation 
conferences. 

Teachers' comments — Theme: Degree of Helpfulness 

• It helps to get useful strategies for the lesson. 

• During preobservation discussion I have a chance to present my ideas. 

• Essential in order to discuss, clarify, or express what areas are being observed. 
Specifically teaching strategies, interaction with students, technology. 

• Useless 

Teachers' comments — Theme: Supervisor's Role 

• Chair's advice is always helpful. 

• My supervisor always listens first and adapts the lesson plan to my teaching 
style. 
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• It is important to know what the chair will focus on. 

Teachers' comments — Theme: Non-conduct of Pre-observation conferences 

• Would be helpful if it happened. 

• Would be helpful but does the supervisor have time? 

• We usually do not have a discussion before my supervisor's visit. 

• Not many preobservation discussions were held with my supervisor. 

• Usually don't have time for it. 

Supervisors agreed that preobservation conferences were beneficial. Teachers, on 
the other hand, were not as convinced of the degree of usefulness of the preobservation 
conference, and, in fact, some teachers reported that the preobservation conferences did 
not usually take place. Teachers additionally commented on the role of the supervisor in 
preobservation conferences, and four of the comments addressed the problem of the 
supervisor not having time to do the preobservation conferences. 

SET was the other teacher-evaluation strategy that showed large differences in 
means and standard deviations between teachers and supervisors. In the following section 
both quantitative and qualitative data are presented to illustrate the differences in 
perspectives between supervisors and foreign-language teachers. 
Student Evaluations: I/ESQs (Interim/End of Course Questionnaires) 

SET was the second evaluation strategy to be examined in more detail. The means 
and standard deviations for supervisors (M = 4.1 1, SD = .33) and teachers (M = 3.55, 
SD = 1.19) were reviewed. SET, specifically I/ESQs, was viewed differently by 
supervisors and teachers. 
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As can be seen in Table 1 1, all of the supervisors (100%) strongly agreed or 
agreed that the I/ESQs are effective as a teacher-evaluation strategy that contributes 
positively to the teachers' professional development. In contrast, Table 11 shows that 
teachers were not as convinced of the effectiveness of I/ESQs. 

Table 11 

Crosstabulation of Supervisors ' and Teachers ' Perspectives on the Effectiveness of 
Student Evaluations 

Student evaluations (I/ESQs) 
Strongly disagree Disagree Undecided Agree Strongly agree Total 





/ 


% 


/ 


% 


/ 


% 


/ 


% 


/ 


% 


Supervisor 


0 


0 


0 


0 


0 


0 


8 


89 


1 


11 


Teacher 


8 


6 


20 


15 


30 


22 


42 


31 


34 


25 



9 

134 



Note. I/ESQ = Interim/End of course Student Questionnaires 



Fifty- six percent of the teachers who responded reported that they strongly agreed 
or agreed that the I/ESQs were effective evaluation strategies that contribute positively to 
their professional development. It must be noted, however, that 28 teachers (21%) 
strongly disagreed or disagreed that I/ESQs are effective as a teacher evaluation strategy. 
Clearly, there is a large difference in the perspectives of supervisors and teachers. 
Furthermore, 22% of the teachers who responded indicated that they were undecided 
regarding the effectiveness of this teacher-evaluation strategy. 

An independent samples t test was conducted and the results revealed that 
p = .001, with a = .05. Again the null hypothesis was rejected in favor of the alternative 
hypothesis. Thus, there was a statistically significant difference between the perspectives 
of teachers and supervisors regarding the student evaluations in teacher evaluation 
(tu\ = 1-40, p = .001). Cohen's d was .48, a moderate effect size. The results of the t tests 
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should be considered cautiously due to the large differences in sample sizes between 
supervisors in = 9) and teachers (n = 134) and the compounding of Type I error when 
conducting multiple t tests. In order to gain additional insight into the views of the 
respondents regarding student evaluation of teaching, open-ended data are presented for 
both supervisors and teachers. Supervisors' written comments are presented initially, 
followed by teachers' written comments. 

Supervisors ' open-ended responses: Student evaluations (I/ESQs). 

Seven of the nine (78%) supervisors provided comments regarding student 
evaluations. All seven of the supervisors who wrote comments strongly agreed or agreed 
that student evaluations are effective in promoting professional development. Two 
themes were identified for supervisors' comments on student evaluations: Identification 
of Patterns and Awareness. Sample comments that exemplify respondents' perspectives 
are provided. 

Supervisors' Comments — Theme: Identification of Patterns 

• Especially to identify patterns observed across ESQs [End of Course Student 
Questionnaires] /ISQs [Interim Student Questionnaires]. 

• Gives another perspective on teaching performance. Can see patterns, areas of 
concerns. 

Supervisors' Comments — Theme: Awareness 

• It may raise awareness and perform correction if needed. 

• Used effectively for reflection. 

• Give's [sic] idea on teacher's planning 



Teachers ' open-ended responses: Student evaluations (I/ESQs). 

Forty-six (34%) of the 134 teachers who responded to this question provided 
comments. The extent of agreement among teachers who provided comments was: 21 
(46%) strongly agreed or agreed, 11 (24%) strongly disagreed or disagreed, and 14 (30%) 
were undecided. Teachers' open-ended comments were categorized into three primary 
themes: Degree of Usefulness, Subjectivity, and Perception of Students Taking Revenge. 

Teachers' Comments — Theme: Degree of Usefulness 

• Gives me an idea of what I do well and in which areas I need to change some 
things. 

• Students' I/ESQs teach me how to be a better teacher, person, manager, and 
just human being. 

• It is helpful to know others' opinions 

It is one of the most important evaluation aspects for me. 
Teachers' Comments — Theme: Subjectivity of Student Evaluations 

• Sometimes it's very subjective and influenced by the students' performance. 

• Not always objective, very often it is written without argumentation. 

• Students' comments might be inaccurate and biased, yet good teachers learn 
from them and adjust/improve teaching. 

Although I believe that students are sometimes not objective. 

• Student evaluations are constructive, however, known to be biased. They 
should not be considered as the whole of an evaluation, but as a feedback 
instrument. 
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• I am not sure, sometimes the comments are very personal and not totally 
objective about a teacher. 

Teachers' Comments — Theme: Students Taking Revenge 

• Sometimes those who don't do well in the course take revenge and write 
nonsense. For this reason, I think their feedback is sometimes unreliable. 

• Students who are counseled take their revenge. 

• Though it has its positive moments sometimes students use it as a revenge 
weapon aimed at the teacher whom they dislike and then it loses its objectivity 
and purpose. 

• Should not be used as a forum to insult teachers. There should be a part where 
students could state what they will do to help teachers succeed in their mission. 

• I encountered "payback" time by students (for writing reports on unfinished 
homework) and other reasons. 

As in the case of previously identified teacher-evaluation strategies, the 
supervisors' comments were unanimously positive. Numerous teachers provided 
comments that relate to the benefits that can be derived from student evaluation of 
teaching; however, teachers expressed negative comments regarding the subjectivity of 
the student evaluations of teaching and the perceptions they have of students taking 
revenge against teachers during these evaluations. Most of the teachers' comments, 24 
out of 46 (52%) were related to the subjectivity of student evaluations. 

Whereas the previous section contained data that addressed the effectiveness of 
the teacher-evaluation strategies, the following section includes data that addressed the 
effectiveness of the evaluation process. The TES, Part II of both versions of the survey, 
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asked the participants to respond to 17 items using a Likert-type scale to identify the 
extent to which they agreed with the statements. 

Effectiveness of the Evaluation Process 

To examine DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, the means and standard deviations regarding respondents' 
perspectives of the effectiveness of the evaluation process are compared. Table 12 shows 
the means and standard deviations for supervisors' responses to Questions 3-19 of the 
TES. The data in Table 12 are presented in rank order with those processes believed to be 
the most effective at the top of the table. Supervisors indicated they strongly believe that 
they provide timely feedback to teachers (M = 4.78, SD = .441). 

The remaining top four evaluation processes that supervisors agreed were 
effective were (a) Supervisors ask teachers to set professional development goals for the 
next teaching year (M = 4.67, SD = .500), (b) Supervisors use the evaluation process to 
help teachers improve their teaching ability (M = 4.67, SD = .500), (c) Supervisors spend 
sufficient time in the classroom to evaluate their teachers' performance (M = 4.67, 
SD = .500), and (d) Supervisors are fair assessors (M = 4.56, SD = .527). 

Table 13 shows the teachers' means and standard deviations for Questions 3-19, 
which measure the effectiveness of various aspects of the teacher evaluation processes. 
The evaluation processes listed in Table 13 are in rank order with the most agreed upon 
processes listed at the top of the table. The item, on the whole, teachers reported was 
most effective in the evaluation process regarded receiving timely feedback from 
supervisors (M = 4.27, SD = .911). The other items that were most agreed upon by 
teachers regarding evaluation-process effectiveness were (a) Teachers do not change their 
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teaching styles when their supervisor is observing their classroom (M = 4.08, 
SD = 1.104); (b) During the evaluation process, teachers set professional-development 
goals for the next teaching year with their supervisor (M = 4.08, SD = .926); (c) The 
supervisor is an accurate assessor (M = 4.01, SD = 1.026); and (d) Teachers and their 
supervisor discuss workshops and seminars for the teachers to attend to improve their 
teaching (M = 4.01, SD = 1.00). 

Table 12 

Supervisors ' Perspectives of the Effectiveness of the Evaluation Process 



Evaluation processes 


M 


SD 


Timely feedback 


4.78 


.441 


Professional development goals 


4.67 


.500 


Evaluation to improve teaching 


4.67 


.500 


Supervisor time in classroom 


4.67 


.500 


Supervisor fair assessor 


4.56 


.527 


Evaluation influences methods 


4.44 


.527 


Supervisor listens 


4.44 


.527 


Standards promote teaching 


4.33 


.500 


Discuss seminars 


4.33 


.500 


Productive dialogue 


4.33 


.500 


Supervisor training 


4.33 


.500 


Supervisor accurate assessor 


4.22 


.441 


Evaluation increases confidence 


4.11 


.601 


Clear performance standards 


4.00 


1.225 


Evaluation system could be improved 


3.78 


.667 


Evaluation for nontenured 


3.11 


1.453 


Teacher change teaching style 


2.89 


.928 



139 



Table 13 

Teachers ' Perspectives of the Effectiveness of the Evaluation Process 



Evaluation processes 


M 


SD 


Timely feedback 


All 


.911 


Teacher change teaching style 


4.08 


1.104 


Professional development goals 


4.08 


.926 


Supervisor accurate assessor 


4.01 


1.026 


Discuss seminars 


4.01 


1.007 


Supervisor time in classroom 


4.00 


1.069 


Evaluation to improve teaching 


3.99 


1.100 


Clear performance standards 


3.95 


1.014 


Productive dialogue 


3.93 


1.128 


Evaluation increases confidence 


3.87 


1.107 


Slunprvi cnr trninintr 

O LlL/t^l VIjVJI 11 til 1 11 1 1 


3.86 


1.111 


Supervisor fair assessor 


3.81 


1.058 


Evaluation influences methods 


3.78 


1.059 


Standards promote teaching 


3.72 


1.172 


Evaluation system could be improved 


3.71 


.972 


Supervisor listens 


3.58 


1.139 


Evaluation for nontenured 


2.79 


1.097 



Table 14 includes a comparison of the evaluation processes that supervisors and 
teachers reported to be the most effective. Both groups agreed that the most effective 
aspect of the evaluation process is that timely feedback is provided to teachers by 
supervisors. The only other item shared by all respondents as one of the top five 
components addressed the teacher and supervisor setting professional-development goals 
for the next teaching year. 
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Table 14 

Supervisor and Teacher Ranking of Effectiveness of Evaluation Processes 



Supervisors and teachers 
(n = 143) 



Evaluation process 


Supervisor ranking 


Teacher ranking 


Timely feedback 


1 


1 


Professional development goals 


2 


3 


Evaluation to improve teaching 


3 


7 


Supervisor time in classroom 


4 


6 


Supervisor fair assessor 


5 


12 


Teacher change teaching style 


17 


2 


Supervisor accurate assessor 


12 


4 


Discuss seminars 


9 


5 



Furthermore, as shown in Table 14, supervisors reported that they believe they are 
fair assessors of teachers, as this item was among their top-five evaluation processes. 
Teachers, however, reported this aspect of the evaluation process to be only the 12th most 
important out of 17 processes. A similar disparity occurred with respect to the statement 
that supervisors are accurate assessors, an evaluation process ranked fourth most 
important by teachers, but 12th most important by supervisors. Another process that 
supervisors and teachers viewed differently was the strategy that addressed whether 
teachers changed their teaching style during classroom observations by supervisors. 
Teachers reported that they do not change their teaching style when being observed. This 
aspect of the evaluation process was the second item teachers most agreed with; however, 
supervisors reported that they believe teachers do change their teaching style when being 
observed. The following section addresses the differences in perspectives between 
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supervisors and teachers regarding the effectiveness of various aspects of the evaluation 
process. 

Evaluation-Process Effectiveness Scale 

To further determine the extent to which supervisors' and teachers' perspectives 
differ regarding evaluation processes, the Evaluation Process Effectiveness Scale was 
developed and an independent samples t test was conducted. As shown in Table 15, 
supervisors' responses reveal a higher mean indicating that they view the evaluation 
processes measured by the scale more positively than teachers. 

As noted in Table 15, overall perceptions of the Evaluation Process Effectiveness 
were compared for supervisors (M = 4.1 1, SD = 0.32) and teachers (M = 3.79, SD = 0.72). 
For the Evaluation Process Effectiveness Scale, Levene's Test for equal variances is not 
assumed, and after adjusting for unequal variances, the means for these two groups were 
found to be statistically significantly different (tu = 2.58, p = .022). 

Table 15 

Means and Standard Deviations for Evaluation Process Effectiveness Scale 



Scale 


Supervisor/teacher 


N 


M 


SD 


Evaluation Process 


Supervisor 


9 


4.11 


.322 


Effectiveness 












Teacher 


134 


3.79 


.716 



The difference for the Evaluation Process Effectiveness scale is considered to be 
moderate (Cohen's d = 0.46), which represents a difference of almost one half of a 
pooled standard deviation between two groups. According to Weinberg and Abramovitz 
(2008), Cohen's d measures the effect size independent of sample size. Furthermore, they 
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reported that effect sizes in the d = .50 range are to be considered medium effect sizes. 
Thus, there was a statistically significant difference between supervisors' and teachers' 
perspectives regarding the teacher-evaluation processes. 

Following a review of the supervisors' and teachers' means and standard 
deviations of the various processes, t tests were conducted to determine if there were 
statistically significant differences in the two groups' views. Only those evaluation 
processes that showed large differences in the means and standard deviations of both 
groups were discussed. The following section contains quantitative data regarding 
selected evaluation processes. 
Selected Evaluation Processes 

As previously mentioned, Questions 3-19 of the TES addressed various aspects of 
the teacher-evaluation process. A review of the means and standard deviations of these 
items indicated that numerous items were characterized by relatively large differences. 
These items asked whether (a) Teachers changed their teaching style when being 
observed in their classroom (Question 5), (b) The teachers' performance standards 
promoted better teaching (Question 7), (c) The supervisor was a fair assessor of teachers 
(Question 10), and (d) The supervisor would listen to any changes the teacher might 
suggest for the evaluation process (Question 1 1). To determine if there were any 
statistically significant differences between the supervisors' and teachers' perspectives on 
the four evaluation processes listed in Table 16, t tests for independent samples were 
conducted. The means and standard deviations of each of the questions are reported in 
Table 16. Each of the five items listed in Table 16 are discussed in the next section. The 
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first process to be analyzed addressed the notion of teachers changing teaching styles 
when being observed. 



Table 16 

Means and Standard Deviations of Selected Evaluation Processes 





Supervisors (n = 9) 


Teachers (n 


= 134) 


Teacher evaluation process 


M 


SD 


M 


SD 


5. Teacher changes teaching style when observed 


2.89 


.928 


4.08 


1.10 


7. Performance standards promote better teaching 


4.33 


.500 


3.72 


1.17 


10. Supervisor is fair assessor 


4.56 


.527 


3.81 


1.06 


1 1 . Supervisor would listen to changes in the 


4.44 


.527 


3.58 


1.14 



evaluation process 

Teachers change teaching style. 

It should be noted that Question 5 is worded negatively; that is, the teachers' 
question was, "I do not change my teaching style when my supervisor is observing my 
classroom." The corresponding question on the supervisors' survey was, "My teachers do 
not change their teaching style to meet my expectations when I observe them." The t tests 
revealed that statistically significant differences were found in the means of supervisors 
and teachers regarding their perspectives on this question, (t 141 = -3.17, p = .002). In 
addition, supervisors rated this item over one standard deviation lower than did the 
teachers (i.e., supervisors' standard deviation is .928; teachers' standard deviation is 1.10). 
Cohen's d = -1.09 and t are thus both negative. According to the data, teachers are more 
convinced than their supervisors that they do not change their teaching styles when being 
observed by their supervisor in the teacher-evaluation process. The second evaluation 
process to be examined addressed performance standards promoting better teaching. 
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Performance standards promote better teaching. 

Question 7 addressed whether performance standards used in teacher evaluations 
promote better teaching. All DLIFLC teaching-faculty members have performance 
standards that are used to evaluate the quality of their instruction. A unified set of 
performance standards exist for teachers in all of the undergraduate language schools and 
departments. Supervisors' means and standard deviations for this question were M = 4.33, 
SD = .500, whereas teachers' means and standard deviations for this question were 
M = 3.72, SD = 1.17. In order to determine if there was a statistically significant 
difference between teachers' and supervisors' perspectives on performance standards, t 
tests were conducted. 

Results of the t tests revealed that there is a statistically significant difference in 
the means of supervisors and teachers regarding their perspectives on whether teachers' 
evaluations are based on performance standards that promote better teaching (ti4.9 = 3.125, 
p = .007). Cohen's d was found to be 0.54, which is approximately half of a standard 
deviation and is considered to be a medium effect size. According to the results of this t 
test, supervisors believe more strongly than teachers that evaluations are based on 
performance standards that promote better teaching. The third evaluation process to be 
reviewed addressed the notion of the supervisor being a fair assessor. 

Supervisor is a fair assessor. 

The next question to show a large disparity between teachers' and supervisors' 
means and standard deviations was Question 10, which addressed whether the supervisor 
is a fair assessor of teaching faculty. Supervisors' means and standard deviations for this 
question were, M = 4.56, SD = .527, whereas teachers' means and standard deviations for 
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this question were, M = 3.81, SD = .1.06. In order to determine if there was a statistically 
significant difference between students' and teachers' perspectives regarding Question 10, 
t tests were conducted. 

The results of the t tests indicated that there is a statistically significant difference 
between supervisors' and teachers' perspectives regarding supervisors being fair 
assessors of teachers, {tu\ = 2.10, p = .037). Cohen's d was found to be 0.72, 
approximately three quarters of a standard deviation, which is considered to be a rather 
large effect size. According to the results of the t test and the effect size, supervisors are 
more convinced of their fairness as evaluators of teaching faculty than are teachers. The 
final evaluation process to be reviewed is the notion of the supervisors' willingness to 
listen to changes the teacher might have regarding the evaluation process. 

Supervisors ' willingness to listen. 

The final question to be analyzed as part of the teacher-evaluation-process section 
of the TES is Question 11, which addressed whether the supervisor would be willing to 
listen to changes teachers might suggest for the evaluation process. Supervisors' means 
and standard deviations for this question were M = 4.44, SD = .527, whereas teachers' 
means and standard deviations for this question were M = 3.58, SD = 1.14. In order to 
determine if there is a statistically significant difference between teachers and supervisors 
perspectives regarding this question, t tests were conducted. 

Results of the t tests revealed that there is a statistically significant difference in 
the means of supervisors and teachers regarding their perspectives on this issue 
(t n 7 = 4.28, p = .001). The difference for this question is approximately three quarters of 
a standard deviation and is considered to be larger than a moderate effect size (Cohen's 
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d = 0.77). Supervisors, therefore, are more convinced than teachers that they listen to 
changes their teachers might suggest for the evaluation process. 

Findings for Research Question 2 involved reviewing both descriptive and 
inferential data. Both quantitative and qualitative data were included. As previously 
mentioned, it is important to acknowledge that the inferential findings based on the t tests 
must be considered cautiously due to the uneven sample sizes of the two groups: 
supervisors (n = 9) and teachers (n = 134). Weinberg and Abramowitz (2008) noted that 
independent samples t tests are robust and emphasized that the t test can withstand a good 
deal of deviation from the underlying assumptions, such as homogeneity of variance. It 
was important to investigate the differences between supervisors' and teachers' 
perspectives regarding teacher-evaluation strategies and processes. The researcher 
determined that it was worth risking Type I error, with the knowledge that further 
research by others would shed light on whether there were any false positives in the 
present study. 

Results for Research Question 3 
What are DLIFLC teachers' perspectives regarding the main objective of the 
evaluation process? 

Teachers ' Perceived Objective of the Evaluation Process 
This research question was answered by examining the teacher responses to the 
TES on Questions 20-23. Questions 20 and 21 asked respondents to indicate the extent of 
their agreement using a Likert-type scale: Question 20, "Improving teacher performance 
is an important objective of the evaluation process," and Question 21, "Assessing quality 
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of instruction is an important objective of the evaluation process." The data in Table 17 
show the teachers' means and standard deviations for Questions 20 and 21. 

Table 17 

Teachers ' Perspectives of Objective of the Evaluation Process 





Teachers («= 


=134) 


Evaluation objectives 


M 


SD 


20. Improving teacher performance 


4.16 


.797 


21. Assessing quality of instruction 


4.15 


.897 



Although Research Question 3 asks specifically about teachers' perspectives regarding 
the main objective of the evaluation process, it is important to note that, when compared 
to supervisors' perspectives, teachers' responses reveal larger standard deviations, which 
signifies more diversity in their responses. 

Table 18 shows the frequencies and percentages of responses provided by 
teachers regarding Questions 20 and 21. Regarding Question 20, the majority (85%) of 
the teachers agreed that improving teacher performance is an important objective. Also, 
as can be seen in Question 21, 87% of the teachers strongly agreed or agreed with this 
item, 6% disagreed or strongly disagreed, and 7% were undecided. 



148 



Table 18 

Teachers ' Important Objective of the Evaluation Process 















Teachers (n = 


134) 










Strongly disagree 


Disa| 


;ree 


Undecided 


Agree 


Strongly 


agree 


Total 




/ 


% 


/ 


% 


/ 


% 


/ 


% 


F 


% 




20. Improving 
performance 


1 


.7 


4 


3.0 


15 


11.2 


66 


49.3 


48 


35.8 


134 


21. Assessing 
teaching 


4 


3.0 


4 


3.0 


9 


6.7 


68 


50.7 


49 


36.6 


134 



Questions 22 and 23 allowed respondents the opportunity to select from among 
six choices given to them to answer the questions. Question 22 asked the respondents to 
indicate what the main purpose of the evaluation process was, and Question 23 asked 
them to indicate what the main purpose of the evaluation process should be. The 
following choices were provided: (a) Assess my teaching performance, (b) Establish 
goals for the next school year, (c) Fulfill an administrative requirement, (d) Improve my 
teaching performance, (e) Promote self-reflection on my professional abilities, and 
(f) Determine future monetary compensation (e.g., merit pay). Participants were also 
given the opportunity to provide their own response if they did not agree with any of the 
options presented. 

Teachers ' Perceived Main Purpose of the Evaluation Process 

Table 19 includes data on the various purposes of the evaluation process that were 
reported by teachers. The purposes, with frequencies and percentages, indicated by 
teachers are listed in rank order. 
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Table 19 

Teachers ' Main Purpose of the Evaluation Process Is... 



Teachers (n = 134) 



Main purpose 


/ 


% 


Assess teaching performance 


45 


33.6 


Improve teaching performance 


37 


27.6 


Fulfill an administrative requirement 


30 


22.4 


Determine future monetary compensation (e.g., 
merit pay) 


11 


8.2 


Promote self-reflection on professional abilities 


3 


2.2 


Establish goals for next school year 


1 


0.7 



Note. Some participants did not respond to Question 22; therefore, not all frequencies total 134 and 
percentages in these cases were less than 100. 



Thirty-four percent of the teachers reported that the main purpose of the 
evaluation process was to assess teaching performance and 28% reported that the main 
purpose was to improve the teachers' teaching performance. Almost a quarter of the 
teachers reported that the main purpose of the evaluation process was to fulfill an 
administrative requirement. 

Table 20 contains data that reveal teachers' perspectives regarding what the main 
purpose of the evaluation process should be. Responses are presented in rank order. As 
can be seen in Table 20, 34% of the teachers reported that the main purpose of the 
evaluation process should be to improve teachers' teaching performance. 

The second most popular response shown in Table 20 was that the main purpose 
of the evaluation process should be to promote self-reflection on the teachers' 
professional abilities, (29%). Seventeen percent of the teachers reported that the main 
purpose of the evaluation process should be to assess teachers' teaching performance. No 
teachers indicated that the main purpose should be to fulfill an administrative requirement. 
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Table 20 

Teachers ' Main Purpose of the Evaluation Process Should Be... 



Teachers (n = 1 34) 



Main purpose 


F 


% 


Improve teaching performance 


45 


33.6 


Promote self-reflection on professional abilities 


39 


29.1 


Assess teaching performance 


23 


17.2 


Establish goals for next school year 


8 


6.0 


Determine future monetary compensation (e.g., merit pay) 


8 


6.0 


Fulfill an administrative requirement 


0 


0 



Note. Some participants did not respond to Question 22; therefore, not all frequencies total 134 and 
percentages in these cases were less than 100. 



Table 21, which includes a cross tabulation of the teachers' perspectives regarding 
what the main purpose of evaluation should be and what the main purpose of the 
evaluation is, presents interesting findings. Table 21 also shows that the majority of the 
teachers who responded to Question 22 reported that they believe the main purpose of 
teacher evaluation is to assess their teaching performance. 

Furthermore, as can be seen in Table 21, a cross tabulation of responses to what 
the main purpose is versus what it should be revealed that 30% of the teachers who 
reported that assessing their teaching is the main purpose of the evaluation process stated 
that improving their teaching performance was what the main purpose should be. 
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Table 21 

Crosstabulation of Teachers ' Perspectives on the Main Purpose of Evaluation Process 



Main Purpose Should Be 



Main Purpose Is 


Assess 
teaching 


Establish 
goals 


Improve 
teaching 


Determine future 
Promote self- monetary compensation 
reflection (e.g. merit pay) 


Total 


Assess teaching 


11 


6 


13 


12 


2 


44 


Establish goals 


1 


0 


0 


0 


0 


1 


Fulfill an administrative 
requirement 


8 


1 


3 


10 


4 


26 


Improve teaching 


1 


1 


21 


10 


0 


33 


Promote self-reflection 


0 


0 


1 


2 


0 


3 


Determine future 
monetary compensation 
(e.g., merit pay) 


2 


0 


4 


4 


1 


11 


TOTAL 


23 


8 


42 


38 


7 


118 



Also noteworthy is the finding that 27% of the teachers who reported that the 
main purpose of the evaluation process is to assess their teaching performance indicated 
that promoting self-reflection on their teaching abilities should be the main purpose. 
Furthermore, 30% of the teachers who stated that improving teaching performance is the 
main purpose of the evaluation process also stated that the main purpose should be to 
promote self -reflection on their teaching abilities. 

Of those who reported that fulfilling an administrative requirement is the main 
purpose of the teacher-evaluation process, 38% stated that the main purpose should be to 
promote self -reflection on their teaching abilities. Also interesting in the cross tabulation 
analysis is the fact that 64% of the teachers who reported that the main purpose of the 
evaluation process is to improve teaching also reported that that is precisely what the 
main purpose should be. 
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Results for Research Question 4 

What are DLIFLC supervisors' perspectives regarding the main objective of the 
evaluation process? 

Supervisors ' Perceived Objective of the Evaluation Process 

This fourth research question was answered by examining the supervisors' 
responses to the TES on Questions 20-23. Questions 20 and 21 asked respondents to 
indicate the extent of their agreement on a Likert-type scale: Question 20, "Improving 
teacher performance is an important objective of the evaluation process," and Question 
21, "Assessing quality of instruction is an important objective of the evaluation process." 

Table 22 includes the supervisors' perspectives regarding the objective of the 
evaluation process. As can be seen in Table 22, supervisors have relatively high means 
for both of these questions, which reflects almost unanimous agreement regarding these 
two items as important objectives of the evaluation process. 

Table 22 

Supervisors ' Perspectives of Objective of the Evaluation Process 





Supervisors in 


= 9) 


Evaluation objectives 


M 


SD 


20. Improving teacher performance 


4.89 


.333 


21. Assessing quality of instruction 


4.89 


.333 



Also, the standard deviations for these two questions are relatively small, in particular 
when compared to the standard deviations of the teachers when asked these same 
questions. 
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Table 23 shows the frequencies and percentages of responses provided by 
supervisors regarding Question 20 and Question 21. The data illustrated in Table 23 
indicate the strong agreement expressed by the supervisors. 



Table 23 

Supervisors ' Important Objective of the Evaluation Process 











Supervisors (n 


= 9) 












Strongly disagree 


Disagree 


Undecided 




jree 


Strongly 


agree Total 




/ 


% 


F 


% 


/ 


% 


f 


% 


/ 


% 


20. Improving 
performance 


0 


0 


0 


0 


0 


0 


l 


11.1 


8 


88.9 9 


21. Assessing 
teaching 


0 


0 


0 


0 


0 


0 


l 


11.1 


8 


88.9 9 



Regarding Question 20, 100% of the supervisors strongly agreed or agreed that 
improving teacher performance is an important objective. For Question 21, 100% of the 
supervisors strongly agreed or agreed that assessing teaching is an important objective of 
teacher evaluation. 

Supervisors ' Perceived Main Purpose of the Evaluation Process 

Questions 22 and 23 allowed the supervisor respondents the opportunity to select 
from among six choices given to them to answer the question. Question 22 asked the 
respondents to indicate what the main purpose of the evaluation process was, and 
Question 23 asked them to indicate what the main purpose of the evaluation process 
should be. The following choices were provided: (a) Assess teaching performance, 
(b) Establish goals for the next school year, (c) Fulfill an administrative requirement, 
(d) Improve teaching performance, (e) Promote self-reflection on my teachers' 
professional abilities, and (f) Determine future monetary compensation (e.g., merit pay). 
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Table 24 includes data on the various purposes of the evaluation process as 
reported by supervisors. The purposes are listed in rank order. 



Table 24 

Supervisors ' Main Purpose of the Evaluation Process Is... 





Supervisors in 


= 9) 


Main purpose 


f 


% 


Assess teaching performance 


4 


44.4 


Improve teaching performance 


3 


33.3 


Promote self-reflection on professional abilities 


1 


11.1 


Determine future monetary compensation (e.g., merit pay) 


1 


11.1 


Establish goals for next school year 


0 


0 


Fulfill an administrative requirement 


0 


0 



Table 24 also shows that 44% of the supervisors reported that the main purpose of 
the evaluation process was to assess teaching performance, and 33% reported that the 
main purpose was to improve the teachers' teaching performance. One supervisor (11%) 
reported that the main purpose of the evaluation process was to promote self -reflection on 
teachers' professional ability. Additionally, one supervisor indicated that the main 
purpose of the evaluation objective was to determine future monetary compensation (e.g., 
merit pay). No supervisors reported that the main objective of the evaluation was to 
(a) Establish goals for the next school year, or (b) Fulfill an administrative requirement. 

Table 25 contains data that reveal supervisors' perspectives regarding what the 
main purpose of the evaluation should be. Responses are presented in rank order. 
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Table 25 

Supervisors ' Main Purpose of the Evaluation Process Should Be... 





Supervisors (n 


= 9) 


Main purpose 


/ 


% 


Improve teaching performance 


4 


44.4 


Promote self-reflection on professional abilities 


4 


44.4 


Assess teaching performance 


1 


11.1 


Establish goals for next school year 


0 


0 


Determine future monetary compensation (e.g., merit 
pay) 


0 


0 


Fulfill an administrative requirement 


0 


0 



Four supervisors (44%) reported that the main purpose of the evaluation process 
should be to improve teachers' performance and four (44%) reported that the main 
purpose should be to promote self -reflection. Only one supervisor reported that the main 
purpose should be to assess teachers' teaching performance. No supervisors reported that 
the main objective of the evaluation process should be to (a) Establish goals for next 
school year, (b) Determine future monetary compensation (e.g., merit pay), or (c) Fulfill 
an administrative requirement. It is interesting to note that, as in the case of the teachers, 
44% of the supervisors who responded to Question 22 also reported that they believed the 
main purpose of teacher evaluation is to assess teaching performance. A cross tabulation 
of the supervisors' perspectives regarding what the main purpose of evaluation should be 
and what the main purpose of the evaluation is can be seen in Table 26. 
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Table 26 

Crosstabulation of Supervisors ' Perspectives on the Main Purpose of Evaluation Process 

Main Purpose Should Be 

Assess Improve Promote self- 
Main Purpose Is teaching teaching reflection Total 



Assess teaching 11 2 4 

Improve teaching 0 2 1 3 

Promote self- reflection 0 0 1 1 

Determine future monetary compensation (e.g., merit pay) 0 1 0 1 

TOTAL 14 4 9 



A cross tabulation of responses to what the main purpose is versus what it should 
be revealed that two (50%) of the supervisors who reported that the main purpose is to 
assess teaching stated that promoting self -reflection on teachers' professional abilities 
was what the main purpose should be. One of the supervisors who reported that 
improving teacher performance is the main purpose of teacher evaluation also reported 
that the main purpose should be to promote self-reflection on teachers' professional 
abilities. Two of the supervisors who reported that the main purpose of teacher evaluation 
is to improve teacher performance also stated that the main purpose should be to improve 
teacher performance. Finally, one supervisor reported that the main purpose of the 
evaluation process is to promote self-reflection, and that supervisor further indicated that 
that is precisely what the purpose should be. 

Summary of Major Findings 

This mixed-methods study incorporated responses to a survey, which included 
open-ended comments from DLIFLC supervisor and teacher respondents who work in 
the UEL. The responses and comments provided data regarding the perceptions held by 



157 

the foreign-language teaching faculty and their supervisors about teacher-evaluation 
strategies and processes. 

Research Question 1 examined the evaluation strategies supervisors used most 
frequently to assess teacher performance. Quantitative data were presented to respond to 
this question. Of the 11 strategies, the 143 respondents (nine supervisors and 134 
teachers) indicated that the most used teacher-evaluation strategy was classroom 
observations. The second through fifth most used teacher evaluation strategies were 
(a) student evaluations, such as the I/ESQs, (b) postobservation conferences 
(c) performance reviews, and (d) student feedback from sensing sessions. 

Research Question 2 investigated foreign-language teachers' and supervisors' 
perspectives regarding the evaluation process. This research question further examined 
the extent to which the perspectives of supervisors and teachers differed regarding the 
evaluation strategies and their effectiveness in promoting professional development. 
Supervisors and teachers agreed that the following two strategies were most effective in 
promoting teachers' professional development: postobservation conferences and 
performance reviews. Although classroom observation was not ranked identically by 
supervisors and teachers, this evaluation strategy was among the top five strategies for 
both groups. 

Both quantitative and qualitative data were presented. Respondents' written 
comments were included for the three evaluation strategies (postobservation conferences, 
performance reviews, and classroom observations) identified by both groups as being the 
most effective in promoting professional development. It is also important to note that the 
foreign-language teachers and their supervisors reported that the least effective strategies 
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in promoting teachers' professional development were action research and portfolios. 
Explanations regarding the effectiveness of these two teacher-evaluation strategies were 
provided. 

To determine if there were statistically significant differences between teachers' 
and supervisors' perspectives regarding the evaluation strategies and processes, two 
scales were developed: the Evaluation and Professional Development Scale and the 
Evaluation Process Effectiveness Scale. No statistically significant difference was found 
for the Evaluation and Professional Development Scale; however, a statistically 
significant difference was found for the Evaluation Process Scale. Following a review of 
the descriptive statistics, various evaluation strategies and processes were identified as 
having large differences in means and standard deviations between the groups, and t tests 
for independent samples were conducted. The purpose of these tests was to determine 
whether the difference between the means of the two groups was statistically significant. 
Teachers' and supervisors' open-ended comments provided additional evidence of the 
perspectives of each of these groups regarding teacher-evaluation strategies. The open- 
ended qualitative comments supported and enhanced the quantitative findings of the 
survey results. 

Research Question 3 asked about teachers' perspectives regarding the main 
objective of the teacher-evaluation process. The majority of the teachers agreed that both 
improving performance and assessing the quality of teaching are important objectives of 
the evaluation process and that its main purpose is to assess as well as improve their 
teaching performance. Some teachers (22%) reported that the main purpose of the 
evaluation process was to fulfill an administrative requirement. It is interesting to note 
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that the second largest group of teachers indicated that promoting self-reflection on their 
professional abilities should be the main purpose of teacher evaluation. 

Research Question 4 examined supervisors' perspectives regarding the main 
objective of the teacher-evaluation process. The majority of the supervisors agreed that 
improving teacher performance and assessing quality of instruction were equally 
important as objectives of the evaluation process. The majority of supervisors also 
reported that the main purpose of the evaluation process was to assess teachers' 
performance. Supervisors were equally in agreement that the main purpose of the 
evaluation process should be to improve teaching performance and to promote self- 
reflection on teachers' professional abilities. 

The next chapter begins with a summary of the study, followed by an 
interpretation of the findings. Following the introduction, the chapter contains a 
discussion of the findings. The discussion is then followed by conclusions and 
implications. The chapter concludes with recommendations for educational practice, 
recommendations for future research., concluding thoughts, and reflections of the 
researcher. 
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CHAPTER V 

DISCUSSION, CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS 

Introduction 

This mixed-methods study included both quantitative and qualitative data to 
determine the perspectives of foreign-language teachers and supervisors regarding 
teacher-evaluation strategies and processes they experienced. The study's data were 
based on responses to a self-administered survey that included both closed-and open- 
ended questions. The population for this study consisted of 134 foreign-language teachers 
and nine supervisors who were employed at DLIFLC's UEL. The purpose of this mixed- 
methods study, which employed a cross-sectional survey, was to examine the views of 
DLIFLC foreign-language teachers and their supervisors regarding the teacher-evaluation 
strategies and processes they use and experience. It further explored the role that 
supervisors play as transformational leaders in conducting teacher evaluations. 

This study provided information regarding the perspectives of foreign-language 
teachers and their supervisors regarding specific teacher-evaluation strategies and various 
teacher-evaluation processes. The respondents' comments further shed light on the views 
they had regarding the role of the supervisor in teacher evaluation. The written comments 
provided by the respondents offered insight into the views held by those directly involved 
in the teacher-evaluation process. The survey used in this study was adapted from 
McDougall (2001) to fit the needs of the DLIFLC environment. 

Discussion 

Participants in this study acknowledged that teacher evaluation at DLIFLC's UEL 
contributes to foreign-language teachers' professional development. Participants further 
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acknowledged the positive role their supervisors play in providing feedback and support 
in teacher evaluation. The quantitative data revealed that there were statistically 
significant differences in the views held by foreign-language teachers and supervisors 
regarding some teacher-evaluation strategies and processes. The qualitative data also 
showed that there were differences in the views held by the foreign-language teachers 
and their supervisors. Supervisors' views and comments were more positive than teachers' 
comments. Respondents' written comments additionally indicated areas in the teacher- 
evaluation program that need to be examined and improved. 

The scholarly literature reports that teacher evaluation has two main purposes. 
According to Danielson (2010), the two main purposes are for quality assurance and to 
promote professional development. The DLIFLC teacher-evaluation system includes both 
of these purposes. The DA's Regulation 690-400 (1998) provides the regulatory guidance 
concerning the implementation of the DLIFLC performance-evaluation program. The DA 
regulation describes the objectives of the performance-evaluation system. The summative 
aspect of the evaluation process addresses the accuracy and quality of the evaluation 
system. The DA regulation also addresses professional development, the formative aspect 
of the evaluation system, which is a critical part of the evaluation process (Beerens, 2000; 
Danielson, 2008). 

Whereas, quality assurance in performance evaluation is an important aspect of 
evaluations, the promotion of professional development is emphasized in the research 
literature. For example, the Personnel Evaluation Standards (JCSEE, 2009) include a 
utility standard that addresses follow-up and professional development. The utility 
standard also includes reference to the fact that evaluations should help teachers improve 
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their performance. An important aspect of the inclusion of utility in personnel evaluations 
is that it encourages evaluatees to be actively involved in their own professional growth, 
which helps create trusting and respectful relationships between evaluatees and their 
evaluators. This study highlighted the contribution that the DLIFLC evaluation system 
makes to the foreign-language teachers' professional development. 

This study additionally explored the views of foreign-language teachers and their 
supervisors regarding teacher evaluation in the light of the theoretical rationale, 
transformational leadership, which advocates that leaders help followers grow and 
develop (Bass & Riggio, 2006). The leader or supervisor plays a critical role in teacher 
evaluation by guiding teachers in their professional development. The results of the 
closed- and open-ended questions posed to the respondents provided the insight and 
additional information that was needed to begin to fill the gap that had been identified in 
the literature. The study provided research regarding teacher evaluation in the context of 
the federal government's primary language school, DLIFLC, which is an educational and 
research institution sponsored by the Department of Defense. 

The following research questions were identified for this study: 

1. What evaluation strategies do DLIFLC supervisors most frequently use to 
assess teacher performance? 

2. What are DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, and to what extent do the perspectives of supervisors and 
teachers differ regarding the evaluation strategies and their effectiveness in 
promoting professional development? 
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3. What are DLIFLC teachers' perspectives regarding the main objective of the 
evaluation process? 

4. What are DLIFLC supervisors' perspectives regarding the main objective of 
the evaluation process? 

Research Question 1 

What evaluation strategies do DLIFLC supervisors most frequently use to assess 
teacher performance? 

This research question examined the responses of teachers and supervisors 
regarding the evaluation strategies that were used in teacher evaluation. Teachers and 
supervisors reported that the top three teacher-evaluation strategies they used or 
experienced were classroom observation, SET, and postobservation conferences. As was 
reported in Chapter IV of this study, supervisors and teachers were unanimous in their 
responses regarding classroom observation, a traditional teacher-evaluation strategy. One 
hundred percent of the respondents indicated that this was the primary teacher-evaluation 
strategy they used or experienced. The second most frequently used or experienced 
teacher-evaluation strategy was SET, which is considered to be a widely used source of 
information for teacher evaluation (Seldin, 1999). Postobservation conferences were the 
third most frequently used teacher-evaluation strategy. Classroom observations, including 
preobservation conferences, are discussed first, followed by SET and postobservation 
conferences. 

The 100% agreement that classroom observations are the most often used teacher- 
evaluation strategy is consistent with the research literature on the popularity of 
classroom observation in teacher evaluation. Classroom observation has been referred to 
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as the "centerpiece of evaluation" (Danielson, 2008, p. 69) and is found to be the most 
common form of teacher evaluation (Peterson, 2000). 

Supervisors at DLIFLC, and in particular the supervisors in UEL, receive training 
in classroom observation for professional development. All UEL supervisors are required 
to attend a professional-development workshop that describes the steps involved in 
conducting effective classroom observations. The principal goal of DLIFLC and UEL 
observation processes is to promote the development of the foreign-language teachers to 
improve classroom performance. Acheson and Gall (1997) outlined the elements of 
clinical supervision, which include a three-step observation cycle process. The clinical- 
supervision approach is the classroom observation approach adopted by UEL supervisors. 

The classroom-observation process includes the observation and pre- and 
postobservation discussions between the foreign-language teacher and the supervisor 
(observer). The preobservation conference is an important step in the observation cycle 
because it provides the observer an opportunity to learn about the type of feedback the 
teacher desires. However, the TES survey results indicated that only 54% of the 
respondents (n = 143) reported that they used or experienced preobservation conferences. 
The findings of this study, related to the limited use of preobservation conferences, are in 
contrast to the research literature. For example, one teacher commented that 
preobservation conferences were "useless" and another teacher wrote: "We usually do 
not have a discussion before my supervisor's visit." Beerens (2000) highlighted the 
importance of involving teachers in the teacher-evaluation process; and preobservation 
conferences would be perfect examples of professional conversations that involve 
teachers and their supervisors who will observe them. 
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SET was the second most commonly used evaluation strategy. Ninety-nine 
percent of respondents, including teachers and supervisors, reported that SET was 
extensively used as part of the evaluation process. The scholarly literature on the subject 
of SET confirmed that information collected from student surveys can be extremely 
valuable to teachers to help them improve their performance (Danielson, 2008). All 
language programs at DLIFLC incorporate student feedback into the performance- 
appraisal process. Supervisors review the student feedback with teachers to determine 
which aspects of the students' comments are valid and appropriate for faculty to consider, 
in order for teachers to improve their instructional effectiveness of the foreign-language 
they teach. 

The vast use of SET as part of teacher evaluation was reported in the literature 
(Barth, 2008; Pallett, 2006; Pounder, 2007). One cautionary note that must be highlighted 
is that SET has often been the single method of obtaining data for teacher evaluations 
(Seldin, 1993, 1999). Related to this cautionary note, one teacher in this study reported 
the following: "[Student evaluations of teaching] should not be considered as the whole 
of an evaluation, but as a feedback instrument." UEL supervisors, however, are asked to 
include multiple sources of data in performance evaluations, which is the recommended 
approach to achieving effectiveness in teacher evaluations (Peterson, 2000; Seldin, 1999; 
Stronge et al., 2008). 

The third teacher-evaluation strategy identified by teachers and supervisors as 
being widely used is postobservation conferences. In contrast to preobservations, 94% of 
respondents reported that they used or experienced postobservation conferences. The 
postobservation conference generally includes a review of the data obtained during the 
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classroom observation and an opportunity for the teacher to reflect and comment. A 
supervisor comment exemplifies, "It leads to self-reflection, potentially resulting in self- 
directed growth." This important third step in the clinical supervision or observation 
cycle should encourage the teacher to analyze the data collected by the individual who 
observed the class. It is also during this phase of the observation cycle that the teacher 
and supervisor develop follow-up plans for the next observation (Zepeda, 2006). 

Based on the data obtained in response to this first research question, UEL 
supervisors are implementing behaviors reflective of transformational leadership. 
Postobservation conferences were identified by both teachers and supervisors as being a 
teacher-evaluation strategy that is frequently used. These conferences provide 
opportunities for dialogue between leaders and followers regarding the followers' growth 
and development (Bass & Riggio, 2006). Supervisors are, thus, engaging the teachers in 
discussions intended to motivate them and to positively impact their work performance. 

Research Question 2 

What are DLIFLC teachers' and supervisors' perspectives regarding the 
evaluation process, and to what extent do the perspectives of supervisors and teachers 
differ regarding the evaluation strategies and their effectiveness in promoting 
professional development? 
Effectiveness of Teacher-Evaluation Strategies. 

The second research question examined the perspectives of teachers and 
supervisors regarding the evaluation strategies identified and their effectiveness in 
promoting professional development. This question additionally asked about teachers' 
and supervisors' views regarding the evaluation processes. Results of the survey used in 
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this study indicated that supervisors and teachers found postobservation conferences and 
performance reviews to be the most effective teacher-evaluation strategies. 

Performance reviews, the second most effective teacher-evaluation strategy, 
consist of the teacher and supervisor reviewing the teacher's performance. Generally, the 
teachers receive feedback on various aspects of their job. UEL teachers and supervisors 
normally participate in three to four performance reviews per year. Performance reviews 
consist of the teacher and supervisor reviewing the teacher's performance standards and 
discussing the teacher's strengths and weaknesses. Additionally, the teacher's 
professional-development plans are discussed and documented in the teachers' Individual 
Development Plan (IDP). 

As previously reported, the TES included both closed- and open-ended questions. 
The written comments provided by the respondents offered additional insight into the 
views of the teachers and supervisors regarding various aspects of the teacher-evaluation 
strategies and processes they experience. The following section consists of selected 
comments regarding postobservation conferences, performance reviews, and classroom 
observations, which illustrate the perspectives held by the survey respondents. All 
supervisors' written comments regarding postobservation conferences and performance 
reviews were positive. 

Regarding postobservation conferences, one supervisor offered the following: 
"Teacher shows the ability to evaluate his own lesson, to show the willingness to grow 
and improve." This comment is an example of the type of outcome that leads to 
professional growth, which is desired as part of teacher evaluation. Another supervisor 
reported, "To have opportunities to discuss feedback and give own perspective on lesson." 
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The supervisors' comments reflected the themes of growth and development which 
represent one of the purposes of effective teacher evaluation, that is, teachers' 
professional development (Beerens, 2000). These comments are also reflective of 
individual consideration, one of the key characteristics of transformational leadership. 
Bass (2000) reported that giving feedback is an important behavior that leaders carry out 
in their roles as coaches and mentors. 

Teachers' comments regarding postobservation conferences were mostly positive. 
One teacher commented that "It helps to get feedback orally and in writing." Another 
teacher wrote that, "It is important to have another perspective of what's happening in the 
classroom as well as feedback with suggestions to improve." A negative comment 
provided by a teacher was: "Too critical and subjective." The comments provided by 
these supervisors and teachers were mostly positive and indicate that congruence exists 
with respect to the perspectives held by supervisors and teachers regarding the role of 
feedback in postobservation conferences. As previously reported in the findings section 
of this study, teachers indicated that the post-observation conferences were "great 
learning experiences" and another indicated that they "provide good communication." 
These comments reflect UEL supervisors' roles as transformational leaders. That is, the 
supervisors in this study were mostly recognized for helping their teachers grow and 
develop as a result of providing feedback to foreign-language teachers. 

Based on a review of the qualitative data generated from the responses to the 
open-ended questions, it is clear that UEL teachers' and supervisors' views on the 
importance of feedback in teacher evaluation is also consistent with what is reported in 
the literature. For example, one teacher stated "It's always useful to hear someone else's 
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professional opinion about your work." Beerens (2000) highlighted that constructive 
feedback is important for the purpose of helping a teacher grow professionally. Although 
London (1997) stressed that individuals are sometimes hesitant to receive feedback on 
their performance, it is clear that the UEL supervisors and teachers are engaged in the 
process of giving and receiving performance feedback. London, however, further 
reported that feedback can unfortunately be destructive rather than constructive. Only one 
teacher provided a negative comment stating that postobservation conferences were "too 
critical and subjective." This teacher's comment exemplifies the destructive aspect of 
feedback. This negative teacher comment additionally demonstrates the need that exists 
for supervisors to build trust because trust is a building-block of transformational 
leadership (Bass & Avolio, 1993). 

Supervisors' provided only positive comments regarding the effectiveness of 
performance reviews; however, teachers provided both positive and negative comments. 
Teachers' comments regarding performance reviews afford additional glimpses into 
teachers' views regarding the importance of feedback. One teacher expressed a positive 
perspective when writing the following: "I like these reviews no matter how positive or 
critical it [sic] can be; I'd like to grow and develop in my profession." Other teachers 
wrote less positive comments regarding feedback in performance reviews: For example, 
"Supervisor's feedback can be very subjective and does not always reflect the reality." 
The important role that supervisors play when they provide constructive feedback to 
teachers, which contributes to teachers' growth and development, is discussed in the 
literature (London, 1997; Strange, 2006). 



170 

In response to the comments that supervisors' feedback can be subjective and that 
they only look for teachers' weak points, it is relevant to note that Ovando (2005) 
endorsed an approach to giving feedback that included giving a feedback message that 
contained one positive comment as well as one area of concern. Participants in Ovando 's 
qualitative study were principals-in-training who were able to engage in a collaborative 
discussion with the observer. These principals-in-training expressed that they appreciated 
getting the feedback in this manner. These participants included some recommendations 
that relate to the giving of effective feedback. They reported that feedback should be 
specific and be based on observable data. They further wrote that feedback should be 
related to the teacher's professional development. Shinkfield (1994) concurred that 
feedback should be constructive in order to help teachers improve their teaching 
effectiveness and grow professionally. 

Teachers identified classroom observations as the third most effective teacher- 
evaluation strategy. Supervisors rated this strategy as the fourth most effective teacher 
evaluation strategy. Supervisors' comments were all positive and addressed the formative 
and summative aspects of evaluation. One supervisor stated, "Not only for summative 
purposes, but it is a key source of information to assess performance and introduce 
change." Supervisory responses were reflective of the views expressed by scholars who 
have written about formative and summative evaluation. Peterson (2000) contended that 
formative evaluations provide feedback that teachers can use to improve their teaching. 
Danielson and McGreal (2000) concurred that formative judgments are critical to 
teachers' learning and growing processes. 
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Teachers' comments regarding classroom observations consisted of positive and 
negative comments. One teacher wrote: "My strong and weak points can be clarified." 
Other teachers concurred that the feedback they receive from their supervisors, based on 
classroom observations, is helpful to the teachers' growth and development. Regarding 
the negative impact of classroom observations, sample teachers' comments included, 
"Very intimidating part of the job," and "I am usually extremely nervous when being 
observed; therefore, I feel that my performance is never at its best." The teachers' 
comments exemplify the critical role that this valuable evaluation strategy plays in the 
UEL evaluation process. It is important at this point to note that supervisors (M = 4.44, 
SD = .73) in this study were more convinced than the teachers (M = 3.96, SD = 1.08) of 
the benefits of classroom observations. Further, it should be noted that 89% of 
supervisors strongly agreed or agreed that classroom observations were effective in 
promoting professional development; whereas, only 76% of the teachers strongly agreed 
or agreed with the effectiveness of this strategy in promoting professional development. 
Also, 10% of teachers strongly disagreed or disagreed with this item. 

A review of the findings revealed that most teachers wrote positive comments 
regarding the role of their supervisor in teacher evaluation; however, a sufficient number 
of teachers provided negative comments regarding the role of their supervisor and the 
role of feedback in teacher evaluation. Thus, it was important to examine the findings 
regarding the role of the supervisor in teacher evaluation through the theoretical lens of 
transformational leadership. Transformational leadership focuses on the roles and needs 
of followers, and transformational leaders strive to create supportive environments to 
enhance the development and growth of their followers (Bass & Riggio, 2006). 
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The transformational leader is charged with assisting followers in reaching their 
full potential. Individual consideration, one of the four factors that contributes to 
transformational leadership, is exemplified by leaders who listen to their followers, acting 
as coaches or mentors to help their followers grow and develop. Leaders who exhibit 
transformational behaviors acknowledge the individual differences of their followers and 
take the specific steps appropriate for each follower to assist them in their professional 
development (Bass, 2000). The quantitative and qualitative findings demonstrated that 
the supervisors in this study implemented key behaviors reflective of transformational 
leadership such as providing effective feedback. The findings also showed that some 
teachers perceived their supervisors to be critical and the feedback to be subjective. Thus, 
a need for building trust, as a characteristic of transformational leadership, was 
demonstrated in order to provide more effective developmental processes. 

Supervisors' comments regarding postobservation conferences, performance 
reviews, and classroom observations were 100% positive. Additionally, supervisors were 
more positive than teachers in their views regarding the effectiveness of these strategies. 
Teachers' written comments were primarily positive; however, the teachers' comments 
included some negative views regarding the effectiveness of these teacher-evaluation 
strategies. 

Evaluation and Professional-Development Scale 

To further examine the second research question, a scale was developed to 
measure the extent to which the perspectives of supervisors and teachers differed 
regarding evaluation strategies and their effectiveness in promoting professional 
development. Based on independent samples t tests of the Evaluation and Professional 
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Development Scale, no statistically significant difference was found between teachers' 
and supervisors' perspectives regarding the effectiveness of the evaluation strategies. 
After reviewing the means and standard deviations of the teacher-evaluation strategies, 
however, two evaluation strategies were selected for statistical testing: preobservation 
conferences and I/ESQs. Following independent samples t tests, statistically significant 
differences were found for both teacher-evaluation strategies. 

A statistically significant difference was found between teachers and supervisors' 
perspectives regarding the effectiveness of preobservation conferences in promoting 
professional development. Additionally, 100% of the supervisors agreed that 
preobservation conferences were effective as a teacher-evaluation strategy; whereas, only 
48% of teachers reported this strategy to be effective: 16% of teachers strongly disagreed 
or disagreed that this was an effective evaluation strategy and the remainder were 
undecided. 

All written comments provided by the supervisors were positive regarding the 
effectiveness of preobservation conferences; however, teachers' comments revealed the 
positive and negative views they held regarding the effectiveness of this evaluation 
strategy. One teacher reported that preobservations were "useless." Four teacher 
respondents wrote comments that showed that preobservation conferences are not always 
held prior to classroom observations. Unfortunately, some of the teachers in this study are 
missing out on an opportunity to build rapport and to develop a trusting, mentoring 
relationship with their supervisor. Acheson and Gall (1997) highlighted the purpose and 
importance of the preobservation conference. They emphasized that it is during this 
initial phase when the planning of the lesson takes place that trust is developed between 
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the teacher and the supervisor (observer). The trusting relationship is essential between 
employee (teacher) and supervisor because it contributes to the professional growth and 
development of the employee (teacher), which leads to improved performance (Bass & 
Riggio, 2006; Gillespie & Mann, 2004; Goleman, 1998). Furthermore, trust is essential to 
the success of a teacher-evaluation program (Beerens, 2000). 

The second teacher-evaluation strategy that was reviewed was I/ESQs. The results 
of the t test for independent samples found that there was a statistically significant 
difference between supervisors' and teachers' perspectives regarding the effectiveness of 
this teacher-evaluation strategy. It was further determined that 100% of the supervisors 
strongly agreed or agreed that this was an effective evaluation strategy, whereas, only 
56% of the teachers strongly agreed or agreed. Additionally, it should be noted that 21% 
of the teachers strongly disagreed or disagreed with the effectiveness of this strategy in 
promoting professional development. 

Supervisors' written comments confirmed the quantitative data regarding student 
evaluation of teaching. For example, supervisors' who wrote comments provided only 
positive comments. The comments addressed the patterns that they observed in students' 
evaluation comments as well as the fact that students' evaluations raised awareness for 
teachers. Teachers, on the other hand, had both positive and negative things to say about 
student evaluations of teaching: 46% of teachers who wrote comments strongly agreed or 
agreed that this was an effective evaluation strategy and 24% of the teachers who 
commented strongly disagreed or disagreed. Teacher comments that reflect the usefulness 
of SET are: "Gives me an idea of what I do well and in which areas I need to change 
some things." The positive teacher comments complement the findings of numerous 
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scholarly researchers who posited that student evaluation can be valid, reliable, and 
useful in teacher evaluation (Centra, 1993; Peterson & Kauchak, 1982; Stronge & 
Ostrander, 2006; Wilkerson et al., 2000). 

Teachers who viewed SET less positively wrote about the subjectivity inherent in 
SET. One teacher provided a comment that illustrates the students' subjectivity: 
"Sometimes [SET is] very subjective and influenced by the students' performance." 
Additionally, teachers wrote about students taking revenge. One teacher comment 
illustrates: "Sometimes those who don't do well in the course take revenge and write 
nonsense. For this reason, I think their [student] feedback is sometimes unreliable." 
Pounder (2007) concurred that student feedback can be biased and unreliable. Similarly, 
Crumbley et al. (2001) noted that SET, which is generally anonymous, may encourage 
honest feedback, yet it also allows students to punish teachers for a number of reasons 
such as grades students have received. Another teacher wrote that students' comments 
should not be the sole data source for teacher evaluations, but rather should be used as 
feedback for the teachers. This comment is reflective of a theme found in the scholarly 
literature on teacher evaluation. Multiple sources of data are encouraged in order to have 
valid and reliable teacher evaluations (Cashin, 1999; Danielson & McGreal, 2000). 
Effectiveness of the Evaluation Process 

In response to the second research question, evaluation processes were also 
examined. It was interesting to note that supervisors and teachers identified the same 
evaluation process as being the most effective: "The supervisor provides timely feedback." 
Supervisors and teachers also agreed that "The teacher and supervisor set professional 
development goals for the next year." A scale was developed to assess the effectiveness 



176 

of the evaluation process. Results of the independent samples t test based on the scale 
revealed that there was a statistically significant difference between teachers' and 
supervisors' perspectives on the effectiveness of the evaluation processes. 

Following a review of the means and standard deviations for the various teacher- 
evaluation processes, it was evident that there were relatively large differences between 
the means of teachers and supervisors regarding specific items. Following independent 
samples t tests, statistically significant differences were found for the selected four survey 
items that examined various aspects of the teacher-evaluation process. The survey items 
addressed whether (a) In Question 5, the teachers changed teaching styles when being 
observed, (b) In Question 7, the teachers' performance standards promoted better 
teaching, (c) In Question 10, the supervisor was a fair assessor, and (d) In Question 11, 
the supervisor would listen to any changes the teacher might suggest for the evaluation. 

Question 5, on the TES for Teachers was worded as follows: "I do not change my 
teaching style when my supervisor is observing my class." This item addressed the 
question of teachers changing teaching style when being observed and the results of the 
comparison between teachers' and supervisors' perspectives showed a large difference (d 
= -1.09). Supervisors believed more strongly than teachers that teachers do change their 
teaching styles when they are being observed by their supervisor. Interestingly, although 
teachers' perspectives were primarily that they did not change their teaching styles, one 
teacher wrote the following comments: "Teachers teach to ICC (Instructor Certification 
Course) expectations when being observed." This teacher is essentially stating that when 
teachers are observed they teach according to the guidelines provided to them in the 
DLIFLC new-teacher training course, the ICC. 
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The question is whether teachers implement the principles they are taught during 
the ICC when they are not being observed or whether they revert to foreign-language 
teaching methods and techniques that may be less effective. One supervisor provided a 
comment that spoke to the issue of teachers changing their teaching style when being 
observed: "Hands-on knowledge of the person, knowing that it is sometimes artificial (e.g. 
Teachers make special effort to "please" observer - use TL, etc.)." Supervisors and 
teachers are thus aware of the fact that one drawback to scheduled classroom observation 
is that this teacher-evaluation strategy can be artificial because teachers teach the class to 
meet the observer's and the school's expectations (Stronge & Ostrander, 2006). 

Question 7 on the TES addressed the teachers' performance standards. Based on 
the data obtained, supervisors believe more strongly than teachers that teacher 
evaluations are based on performance standards that promote better teaching (d = 0.54 — a 
medium effect size). The year prior to the research study, UEL supervisors had spent a 
considerable amount of time discussing and reviewing teachers' performance standards in 
order to better understand the elements to be observed and evaluated. These discussions 
may have influenced the supervisors to respond that the standards promote better 
teaching. 

Data obtained regarding Question 10, which related to the supervisor being a fair 
assessor, revealed that supervisors are more convinced of their fairness when evaluating 
teaching performance (d = 0.72, a rather large effect size). Thus, teachers were less 
convinced of their supervisors' fairness. The perspective expressed by the teachers might 
be attributable to two factors referenced in the research literature. One is that those 
responsible for conducting teacher evaluations need ongoing training in the difficult task 
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of assessing performance. The other factor that may have bearing on this question is that 
supervisors (i.e., those who conduct teacher evaluations) find themselves in the dual roles 
of being coaches during the formative phase of the evaluation process and judges during 
the summative phase of the evaluation process (Cardno, 1999; Centra, 1993; Peterson et 
al., 2000; Raths & Lyman, 2003). Both of these features of teacher evaluation may have 
contributed to teachers' perspectives regarding supervisor fairness in teacher evaluation. 

As previously mentioned, UEL supervisors had participated in meetings to 
discuss various aspects of teacher evaluation. In addition to discussing the performance 
standards, supervisors discussed fairness and objectivity in teacher evaluation. UEL 
supervisors concurred that one way to strive for fairness and objectivity in evaluating 
teachers was to ensure that multiple sources of data were considered in the teachers' 
evaluations. This practice was confirmed in the research literature (Beerens, 2000; Finch 
et al., 1997; Peterson, 2004; Seldin, 1999; Strange et al., 2008). 

Question 11 on the TES dealt with the notion of supervisors' willingness to listen 
to changes the teacher might suggest for the evaluation process. Teachers were not as 
convinced as supervisors that supervisors would listen to changes teachers might 
recommend for the evaluation process. For this test, a rather large effect size was found: 
Cohen's d = 0.77. In the discussion of Question 1 1, it is important to acknowledge that 
the DA evaluation system is prescribed in DA Regulation 690-400 (1998) and that there 
are aspects of the evaluation process that cannot be changed, such as the components of 
the TAPES described in Subchapter 1-5 of the regulation. 

Examples of components of the TAPES that cannot be changed are (a) the ratings 
of the standards (Excellence, Success, Needs improvement, and Fails), (b) the rating 
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formulas (e.g., the number of standards that must be exceeded to achieve a Success Level 
1), and (c) the overall performance ratings (Success Levels 1-3, Fair, and Unsuccessful). 
It is possible that teachers believe their supervisors are unfair if the teacher is rated less 
than Success Level 1. There are other areas, however, that are not prescribed in the 
regulation, such as the sources of data and how the evidence for teacher evaluation is 
obtained and applied to the teacher evaluation. 

The fact that a significant number of teachers expressed that they do not believe 
their supervisors would listen to any changes they might recommend for the evaluation 
process, suggests there is a need for supervisory training on the key characteristics of 
transformational leadership. Leaders who exemplify traits of transformational leaders 
make connections with their followers to motivate them to reach their full potential and to 
perform beyond their expectations (Bass & Riggio, 2006; Jung et al., 2003). 

Research Question 3 

What are DLJTLC teachers' perspectives regarding the main objective of the 
evaluation process? 

The third research question examined teachers' perspectives regarding the main 
objective of the evaluation process. Eighty-five percent of teachers strongly agreed or 
agreed that improving teacher performance was an important objective of the evaluation 
process. Similarly, 87% of the teachers agreed that assessing quality of instruction was an 
important objective of the evaluation process. These findings complement the research 
literature, which indicates that the two primary purposes of teacher evaluation are for 
quality assurance and for professional development (Andrews, 2004; Beerens, 2000; 
Danielson, 2008; JCSEE, 2009). 
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At DLIFLC, teacher evaluations are used for personnel actions and for quality 
assurance. Teachers' performance evaluations are used for making personnel decisions 
such as when applying for jobs in the Institute, for rank advancement, for merit pay, or 
when competing for tenured positions, for example. Therefore, much emphasis is placed 
on the importance of the performance evaluations. Teachers and supervisors are 
cognizant of the fact that teachers' performance evaluations are used for advancement 
and, thus, the evaluation and the processes involved take on special significance. In 
addition to the quality-assurance aspect of the performance evaluations, DLIFLC teachers 
and their supervisors are asked to collaborate and discuss the teachers' professional- 
development goals. An IDP is prepared jointly by the teacher and supervisor. The IDP 
includes the teachers' professional goals and generally also contains a list of professional- 
development workshops, seminars, or projects the teacher wishes to be involved in during 
the evaluation period. Thus, the uses of teacher evaluation at DLIFLC reflect the 
objectives and uses reported in the scholarly literature on teacher evaluation (Danielson, 
2010; JCSEE, 2009; Natriello, 1990). 

In response to Question 22 on the TES, teachers were given the opportunity to 
select from a list of six purposes of teacher evaluation. When asked to select the main 
purpose of the teacher-evaluation process, 34% of the teachers reported that it was to 
assess teaching performance; whereas, 28% of teachers claimed that the main purpose of 
the teacher-evaluation process was to improve teaching performance. It should be noted 
that 30 (22%) teachers reported that the main purpose of the teacher-evaluation process 
was to "fulfill an administrative requirement." Those teachers who responded that the 



181 

purpose was to fulfill an administrative requirement most likely do not see any other 
value to the evaluation process. 

The fact that 22% of the teachers see no other value to the teacher-evaluation 
process other than to fulfill an administrative requirement reflects a need for more 
dialogue between supervisors and teachers regarding the various teacher-evaluation 
strategies and processes and the purposes of teacher evaluation. As previously reported, 
transformational leaders serve as coaches and mentors who implement individual 
consideration, which consists of giving and accepting feedback as part of growth and 
development (Bass, 2000). 

In Question 23 on the TES, teachers were also asked to indicate what they 
believed the main purpose of the evaluation process should be. Again, teachers were 
given a list of options from which to select. In this case, 34% of teachers reported that the 
main purpose should be to improve teaching performance. Twenty-nine percent of the 
teachers selected "promote self -reflection on professional abilities" as the second most 
popular response to what the main purpose of teacher evaluation should be. The third 
response (17%) selected was "assess teaching performance." Thus, it is interesting to 
observe that teachers desire that their evaluations promote self-reflection. 

A review of the scholarly literature found that two specific teacher-evaluation 
strategies are most identified with contributing to self-reflection. Those strategies are 
portfolios and action research. Findings from this study, however, revealed that 
supervisors and teachers agreed that these two evaluation strategies were the least used in 
teacher evaluation. Only faculty who hold the ranks of Associate Professors and 
Professors are required to conduct action research. The requirement is documented in the 
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DLIFLC teachers' performance standards. Other teachers are encouraged to conduct 
action research; but they are not required to do so. Whereas, respondents' reported that 
portfolios as teacher evaluation were among the least effective teacher-evaluation 
strategies, it must be clarified that this is due to the fact that teachers are not required to 
prepare portfolios. Respondents, therefore, rated this teacher evaluation strategy low in its 
effectiveness as a strategy to promote professional development. Further, because 
teachers at DLIFLC must teach a high number of teaching hours, they reported that they 
would not have time to prepare portfolios as part of their annual evaluations. Portfolios 
were, nevertheless, included in the survey because the scholarly literature placed much 
emphasis on portfolios for self-evaluation in order to promote professional growth and 
development. 

The low ranking of these two important evaluation strategies may be due to the 
fact that only a relatively small number of teachers and supervisors are required to do 
action research. Also, although many UEL teachers saw the value in conducting action 
research and preparing portfolios, they expressed that due to the heavy teaching load, 
they did not have time to engage in these meaningful practices. Scholars maintained that 
these two strategies contributed to teachers' professional growth as a result of self- 
reflection (Glanz, 1999; Murphey & Yaode, 2007; Sagor, 2000; Zepeda, 2002). 

Research Question 4 

What are DLIFLC supervisors' perspectives regarding the main objective of the 
evaluation process? 

The fourth research question examined supervisors' perspectives on the main 
objective of the evaluation process. All nine supervisors (100%) strongly agreed or 
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agreed that improving teaching performance was an important objective of the evaluation 
process. Similarly, 100% of the supervisors also agreed that assessing quality of 
instruction was an important objective of the evaluation process. With respect to the 
purposes of teacher evaluation, the findings in this study confirmed what is reported in 
the scholarly literature regarding the main purposes of teacher evaluation. That is, teacher 
evaluation at DLIFLC's UEL is used for the purpose of assessing the quality of teaching 
and to improve teaching (Danielson, 2010). 

It should be noted that there is an interesting difference when one compares the 
supervisors' and teachers' responses regarding the main objective of the teacher- 
evaluation process — improving teacher performance and assessing quality of instruction. 
The data revealed that supervisors were much more in accord with their responses on 
these two items (SD = .33). 

When asked to report what the main purpose of the evaluation process is, four 
(44%) of the supervisors indicated that the main purpose of the evaluation process is to 
assess teaching performance. This response was similar to the teachers' response. That is, 
teachers also reported that assessing teaching performance was the main purpose of 
teacher evaluations. Three (33%) of the supervisors reported that improving teaching 
performance was the main purpose of the teacher-evaluation process. 

When supervisors were asked to report what the main purpose of the evaluation 
process should be, four (44%) of the supervisors reported that the main purpose should be 
to improve teaching performance. It is noteworthy that four (44%) other supervisors 
reported, in response to this question, that the main purpose of teacher evaluation should 
be to promote self -reflection on professional abilities. Only one supervisor reported that 
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the main purpose should be to assess teaching performance. Additionally, 29% of the 
teachers also selected promotion of self -reflection on professional abilities as their second 
most popular response to this question. 

UEL survey participants in this study acknowledged that they are aware of the 
value of self -reflection on professional abilities. Unfortunately, however, due to the 
heavy teaching load and many additional requirements placed on the teachers, many are 
unable to participate in the creation of portfolios, a teacher-evaluation strategy that is 
touted as being a key tool to promote self-reflection and professional growth. Through 
the use, however, of classroom observations, postobservation conferences, performance 
reviews and even SET, teachers are encouraged to self-reflect in order to improve their 
teaching performance. The three evaluation strategies reported by participants as being 
effective all involve conversation and dialogue. In this way, UEL supervisors exhibited 
an important trait of transformational leadership — individual consideration — which 
involves providing effective feedback (Bass & Riggio, 2006). 

Conclusions 

Teacher evaluation is believed to be one of the most complex issues in education 
and is considered to be essential for understanding effective teaching and learning 
(Beerens, 2000; Danielson & McGreal, 2000). The main purposes for teacher evaluation 
are to ensure teacher quality and to promote teacher learning (Danielson, 2008). This 
study has examined the strategies and processes of the teacher-evaluation system 
implemented at DLIFLC's UEL. It has also explored the role of supervisors as 
transformational leaders in the context of teacher evaluation. 
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Based on the quantitative and qualitative data obtained from the foreign-language 
teachers and their supervisors who served as participants in this study, findings showed 
that of the 1 1 teacher-evaluation strategies identified, classroom observations and student 
evaluation of teaching are the two most used or experienced strategies. The third most 
used or experienced teacher-evaluation strategy is postobservation conferences. Teachers 
and their supervisors also found that performance reviews were widely used or 
experienced. Of these four strategies, the respondents found postobservation conferences 
to be the most effective in promoting professional development. 

Postobservation discussions between teachers and supervisors regarding the 
observed lessons provided teachers with the opportunity to "get feedback orally and in 
writing," as one teacher observed in the written comments. Furthermore, teachers found 
the postobservations to be "great learning experiences." Receiving feedback and 
suggestions regarding teaching were also appreciated by teachers. Supervisors likewise 
expressed a positive view toward postobservation conferences because the conferences 
allowed the supervisors "to have opportunities to discuss feedback and give [their] own 
perspective on lessons." Danielson and McGreal (2000) affirmed that one of the essential 
elements that contributes to professional learning is reflection on practice. Teachers and 
supervisors discussing data collected from an observed class provides an opportunity for 
the teacher to reflect and receive suggestions that can be considered for future classes. As 
previously indicated, the discussions between teachers and their supervisors that lead to 
the teacher reflecting on practice are characteristic of the behaviors exhibited by 
transformational leaders, that is, leaders focusing on feedback and the individual needs of 
their followers (Bass & Riggio, 2006). 
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Acheson and Gall (1997), who wrote about clinical supervision, referred to 
postobservation conferences as feedback conferences. It is during these conferences that 
the supervisor elicits the teachers' perspectives on the class taught after reviewing the 
objective data collected by the supervisor. The teacher is encouraged to take the lead in 
analyzing and interpreting the data. Finally, the teacher is guided to make decisions about 
changes the teacher might make for future classes. McGreal (1983) astutely noted that 
clinical supervision highlights the close relationship that should exist between teacher 
and supervisor and emphasized that this relationship should be collegial and not 
authoritative. McGreal further asserted that the more teachers talk about teaching, the 
more they learn about it. Based on this study's findings, teachers and supervisors are 
actively engaged in professional conversations regarding teaching following classroom 
observations. 

Performance reviews, which consist of teachers and supervisors discussing 
various aspects of teachers' performance, were the second most effective teacher- 
evaluation strategy. Based on the findings of this study, however, teachers expressed 
reservations and some concerns regarding this important teacher-evaluation strategy. The 
performance reviews, much like the postobservation conferences, have as their goal to 
provide teachers with opportunities to receive feedback and reflect on their teaching and 
other related tasks. Whereas, supervisors in this study viewed performance reviews as 
positive experiences that allowed the supervisors the opportunity to learn about the 
teachers' views of themselves as teachers, teachers were not all in agreement that 
performance reviews were effective. 
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One teacher commented that the reviews were "biased" and another teacher wrote 
that if the supervisor is knowledgeable and fair, then the review is "ok, but if not, then it 
means nothing." These negative comments may be due to the fact that some supervisors 
lack training in giving teachers feedback, an important aspect of teacher evaluation. Lack 
of evaluator training is a concern that is expressed in the literature regarding problems 
with teacher evaluation. Teachers who believe that their supervisors are informed and 
trained in conducting performance reviews are more receptive to the feedback the 
supervisors give. The need for administrator training in teacher evaluation is well- 
documented in the research literature (Beerens, 2000; Painter, 2000; Seldin, 2006). 

In addition, teachers in this study were not as convinced of the fairness of their 
supervisors as evaluators as the supervisors were. The issue of evaluator fairness is also 
discussed in the literature. A statistically significant difference was found regarding 
teachers' and supervisors' perspectives regarding fairness of supervisors in teacher 
evaluations. This finding is prevalent in the teacher-evaluation literature (Beerens, 2000; 
Braskamp & Ory, 1994; Centra, 1993; Danielson & McGreal, 2000; Peterson, 2000). 
Danielson and McGreal referred to the perceptions teachers have that the evaluation 
exercise is a "gotcha" (p. 5). That is, teachers sometimes believe that the evaluation is an 
opportunity for the supervisor to find fault with the teachers' performance. Teachers' 
perceptions of supervisors evaluating teachers for the sole purpose of finding fault with 
the teachers' performance is also reflective of a lack of trust. In transformational 
leadership, leaders' and followers' relationships are based on trust so that the leader can 
serve as a mentor or coach for the follower (Avolio & Bass, 1995; Humphreys & Einstein, 
2003). 
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Regarding classroom observations, supervisors and teachers agreed that they were 
effective, with teachers rating them as the third most effective strategy and supervisors 
rating them as the fourth most effective teacher-evaluation strategy. However, teachers 
and supervisors did not agree on one aspect of this important teacher-evaluation strategy. 
Teachers agreed that they do not change their teaching styles when supervisors observe 
their classes. Supervisors, on the other hand, did not agree as strongly with this statement. 
Marshall (2005) discussed the problems with observers giving teachers advanced notice 
that they will be observed. Marshall posited that this contributes to supervisors observing 
classes that are atypical. In this regard, a supervisor noted that teachers' classes are 
artificial because they try to please the observer. One teacher commented that, indeed, 
teachers follow specific guidelines that they are given in the faculty-development 
workshop; however, it is not known if teachers follow these guidelines when they are not 
being observed. 

Supervisors and teachers also did not agree on the effectiveness of preobservation 
conferences. Supervisors reported that these were effective as an evaluation strategy, 
rating them third in their list of effective evaluation strategies. Teachers, on the other 
hand, rated them ninth of 11 strategies, in effectiveness. All supervisors' written 
comments were positive. Teachers' comments revealed that some teachers thought 
preobservation conferences were "useless" and others reported that they do not generally 
have preobservation conferences with their supervisors. The difference in teachers' and 
supervisors' views on this aspect of the observation cycle was also found to be 
statistically significant. 
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Supervisors are required to observe their teachers four times during the evaluation 
year. Some supervisors must evaluate more than 20 teachers annually. It is possible that 
due to time constraints, some supervisors are omitting this valuable, first-step in the 
observation process. It also should be reported that observations may be announced or 
unannounced. An announced observation would require a preobservation conference, 
whereas an unannounced observation would not require a preobservation conference. 

Another teacher-evaluation strategy on which teachers and supervisors did not 
agree was the effectiveness of SET. Supervisors concurred that this was an effective 
teacher-evaluation strategy, whereas only slightly more than 50% of the teachers reported 
that this was an effective strategy. Some teachers provided positive comments regarding 
SET; however, others noted that student comments regarding teaching were often not 
objective. A teacher commented that SET should not represent the only source of data 
when evaluating teachers. This comment reflects the consensus in the literature that there 
should be multiple sources of data in teacher evaluation (Danielson & McGreal, 2000; 
Peterson, 2006b; Seldin, 1999). 

Teachers and supervisors unanimously concurred that teachers receive timely 
feedback from their supervisors and that supervisors spend sufficient time in classrooms 
to evaluate teaching performance. These findings are in compliance with the utility 
standard set forth in the Personnel Evaluation Standards (JCSEE, 2009). The utility 
standard promotes the notion that evaluations should be timely and useful to teachers to 
help them improve their performance. Teachers and supervisors also agreed that they 
work together to set professional-development goals for the next teaching year. This is 
another example of compliance with the Personnel Evaluation Standards, as the utility 
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standard also calls for evaluations to promote professional development. It also reveals a 
behavior reflective of transformational leaders who help their followers by showing 
support and encouraging them to be autonomous in accomplishing their goals (Avolio & 
Bass, 1995). 

Teachers and supervisors in this study agreed that improving teacher performance 
and assessing quality of instruction are important objectives of the teacher-evaluation 
system. These findings complement the purposes specified in the Personnel Evaluations 
Standards, which state that performance evaluations are used to assess the quality of 
teaching and to help teachers improve (JCSEE, 2009). Teachers and supervisors also 
agreed that assessing teaching performance is the main objective of teacher evaluation. 
They additionally agreed that improving teaching performance should be the main 
objective of teacher evaluation. An observation that should be highlighted is that 22% of 
teachers indicated that the main objective of the teacher-evaluation system is to "fulfill an 
administrative requirement." 

Of the group of teachers who reported that the main purpose of their evaluation 
system was to fulfill an administrative requirement, 39% further expressed that the main 
purpose of the evaluation system should be to promote self -reflection on their teaching 
abilities. Teachers' perspectives that the main purpose of teacher evaluation is only to 
fulfill an administrative requirement, should be carefully considered. The results of this 
study complemented McDougall's (2001) findings. That is, teachers in McDougall's 
study also believed that the main purpose of teacher evaluation was to fulfill an 
administrative requirement; and they additionally reported that the main purpose should 
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be to improve their teaching ability. Beerens (2000) reported that teachers did not see the 
benefit of teacher evaluation, characterizing it as being a dead end and not productive. 

Teachers and supervisors in the current study further agreed that the main 
objective of teacher evaluation should be to improve teaching performance. Interestingly, 
both groups selected "promote self -reflection on professional abilities" as their second 
choice in terms of what the main purpose of teacher evaluation should be. Improving 
teaching performance and promoting self-reflection for professional growth and 
development are well-documented as important purposes of teacher evaluation (Bernstein, 
2004; Danielson & McGreal, 2000; JCSEE, 2009). 

This research showed the similarities and differences in perspectives held by 
teachers and their supervisors regarding teacher-evaluation strategies they use or 
experience. Findings also included teachers' and supervisors' perspectives regarding the 
effectiveness of these strategies and various evaluation processes. Additionally, the role 
of the supervisors as transformational leaders who help their followers grow and develop 
was explored. Many of the features of the teacher-evaluation program in the UEL are 
positive and consistent with the guidelines and recommendations found in the literature. 
For example, teacher-evaluation strategies such as postobservation conferences and 
performance reviews seem to be implemented in a manner that promotes teachers' 
professional development; however, there are areas that must be reviewed and adjusted to 
improve the evaluation system. Examples of teacher-evaluation strategies that should be 
examined for more effective implementation are preobservation conferences and SET. 
Aspects of the evaluation process that merit further review are the teacher performance 
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standards, the teachers' perspective that the supervisor is a fair assessor, and supervisors' 
willingness to listen to changes the teacher might suggest for the evaluation process. 

Implications 

The findings from this study have important implications for those involved in 
teacher evaluation in educational institutions in the federal government. This study was 
essential because there were no studies that addressed the effectiveness of the teacher- 
evaluation system at DLIFLC or at educational institutions in the federal government. 
Teachers and administrative personnel who engage in teacher evaluation or who are 
responsible for the implementation of an effective teacher-evaluation system will find 
this study's results and conclusions beneficial. This study contributed to educational 
research in the areas of teacher-evaluation strategies and teacher-evaluation processes. It 
additionally highlighted the important role that leaders who serve as evaluators play in 
ensuring that performance evaluations are effective in promoting professional 
development and growth of the evaluatee. 

The findings of this study have implications for teachers, supervisors, and 
educational and administrative personnel involved in teacher evaluation. The foreign- 
language teachers and supervisors who participated in the study shared their perspectives 
regarding the teacher-evaluation strategies and processes they used or experienced. 
Teachers and supervisors agreed that the four most used or experienced evaluation 
strategies are classroom observation, SET, postobservation conferences, and performance 
reviews. Therefore, it will be important for both teachers and supervisors to fully 
understand the benefits and drawbacks of each evaluation strategy. Although, the 
research literature confirmed the importance of self-evaluation as a form of teacher 
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evaluation (Airasian & Gullickson, 1997), this study's results found that self-evaluation, 
the use of portfolios, and action research were not used as frequently as the 
abovementioned evaluation strategies. It will, therefore, be important for those 
responsible for evaluating the Institute's and UEL's evaluation practices to consider the 
benefits of these evaluation strategies. 

Teachers and supervisors also conveyed their viewpoints regarding the 
effectiveness of the various teacher-evaluation strategies and evaluation processes they 
experience. The study's findings, thus, have implications for teachers, supervisors, and 
those individuals in leadership positions who may be able to make any necessary 
adjustments to the current teacher-evaluation system. Preobservation conferences, for 
example, were found to be effective by supervisors; however, teachers did not find this 
evaluation strategy to be as effective as did supervisors. One reason for this difference 
may be that some supervisors are not conducting preobservation conferences due to time 
constraints. Research studies on the phases of classroom observation highlight the 
importance of preobservation conferences as essential to helping the teacher with lesson 
planning (Zepeda, 2006). 

SET was another evaluation strategy that supervisors found to be very effective, 
whereas teachers were not as positive about the effectiveness of this evaluation strategy. 
The use of SET in teacher evaluation is controversial and the research literature includes 
arguments for and against its use in teacher evaluation (Centra, 1993; Marincovich, 1999; 
Ngware & Ndirangu, 2005; Oliver-Hoyo, 2008; Seldin, 1999). Although 56% of the 
teachers in this study strongly agreed or agreed with the use of SET in teacher evaluation; 
21% of the teachers reported that they strongly disagreed or disagreed with its use. 
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Teachers further expressed concern that student feedback was subjective and that it 
should not be the sole source of information for their performance evaluations. These 
findings most certainly have implications for those responsible for the implementation of 
an effective teacher-evaluation system. That is, DLIFLC administrators and others 
responsible for implementing teacher-evaluation programs must consider the 
controversial nature of SET and ensure it is used in a constructive manner. 

This study's findings also have implications regarding this study's theoretical 
framework, transformational leadership, and the role of supervisors in teacher evaluation. 
Most teachers reported that they received helpful feedback from the various teacher- 
evaluation strategies they experienced. They appreciated learning about their strengths 
and weaknesses and further expressed that the feedback helped them improve the quality 
of their teaching. Transformational leadership encourages individual consideration to 
assist followers in their growth and development (Bass, 2000). 

Transformational leaders additionally provide supporting and trusting 
environments and serve as mentors and coaches. Some teachers in this study pointed out, 
however, that they found their supervisors' feedback to be subjective and biased, and that 
the feedback only served the purpose to find the teacher's weak points. These comments 
serve as evidence that some teachers do not perceive their supervisors to be supportive 
and encouraging guides or coaches. The view of a supervisor who only looks for the 
teachers' weak points runs counter to the principles of transformational leadership. 
Teachers' written comments provide valuable evidence of teachers' perspectives of the 
role that their supervisors play in the implementation of the teacher-evaluation program. 
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Recommendations for Professional Practice 
The results of this study may serve as a catalyst for various projects related to the 
assessment, maintenance, and improvement of teacher evaluation systems at DLIFLC, in 
UEL in particular, and at other federal institutions, especially those that use the Army's 
evaluation program, the TAPES. With respect to the assessment of teacher-evaluation 
programs, it would be important for those responsible for the oversight of teacher- 
evaluation programs to conduct a needs assessment of their teacher-evaluation programs 
to determine the areas where the program is functioning well and where improvement 
might be needed. Whereas a needs assessment will identify the strengths and weaknesses 
of teacher-evaluation programs, certainly the findings contained in this study will also 
contribute to the assessment of teacher-evaluation programs. The findings provide insight 
into the perspectives of foreign-language teachers and supervisors regarding teacher 
evaluation; and their views may be beneficial to others as they evaluate their teacher- 
evaluation programs. 

Furthermore, the results of this study will contribute to specific areas of the 
DLIFLC and UEL teacher-evaluation systems. For example, based on the findings of this 
study, there is a need for separate training programs for teachers and supervisors 
regarding the purposes of teacher evaluation. The teachers' training program should 
include information regarding the various strategies used in teacher evaluation and should 
highlight the benefits and drawbacks of each. The role of SET, in particular, should be 
reviewed in both the teacher and supervisor workshops. There is abundant research 
literature on the advantages and disadvantages of SET in teacher evaluation (Centra, 
1993; Marincovich, 1999). Teachers should be given an opportunity to review this rich 
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literature and to discuss how this valuable source of data might best be used for teacher 
growth and development. 

An important component of these training programs would be the importance of 
self -reflection for professional growth and development. The topics of self-reflection and 
self-evaluation should be a major component of the teacher-training program to ensure 
that teachers are made aware of the benefits of these practices in teacher evaluation 
(Airasian & Gullickson, 1997; Beerens, 2000; Braskamp & Ory, 1994; Danielson & 
McGreal, 2000; Lord, 2009). 

An additional component for the teacher and supervisor workshops should be the 
role of the performance standards to help improve teaching performance. Teachers and 
supervisors review and sign the teachers' performance standards at the beginning of the 
evaluation year; however, it is not known whether teachers fully understand the purpose 
of the performance standards. Teachers should be made aware of the importance of the 
performance standards and explicit connections should be made between the specific 
standards and how they each relate to improved performance. 

Although DLIFLC teachers are required to teach a rather high number of 
instructional hours and additionally spend hours preparing their classes and providing 
advising or counseling to students, it will be important to ensure that teachers are 
informed of the benefits of teacher-evaluation strategies that contribute to self-reflection. 
In particular, the benefits of portfolios and action research should be included in the 
training programs on teacher evaluation. Portfolios might be particularly useful to 
nontenured teachers as part of their professional development and could serve to assist 
them in preparing for tenure competition and rank advancement. It will also be important 
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to clarify to teachers which aspects of the evaluation process cannot be changed. 
Furthermore, it will be important to query teachers regarding specific recommendations 
they may have to make their evaluations more effective. 

The supervisors' training program should additionally include segments that 
would address the following components: (a) the role of feedback in teacher evaluation 
and its contribution to the teachers' growth and development (London, 1997; Stronge, 
2006), (b) the role of the supervisor in formative and summative evaluation, (c) the 
advantages and disadvantages of the various teacher-evaluation strategies they use, and 
(d) the use of multiple sources of data and how each of the various sources might be used. 
It will also be important to include in the supervisors' workshop on teacher evaluation, a 
segment on the characteristics of transformational leadership and how this leadership 
style can be tied to effective teacher evaluation. In light of some of the feedback provided 
by teachers that discussions regarding the teachers' performance "are never a two-way 
street" and that the supervisors' feedback is sometimes subjective, it will be important to 
include a segment that will highlight the importance of dialogue, specifically two-way 
communication regarding the teachers' strengths and weaknesses. Bass (2008) discusses 
the role of leaders as mentors and coaches who recognize the individual differences in 
their followers. 

The teacher and supervisory workshops on teacher evaluation should additionally 
include information on Danielson's (1996, 2008) framework for teaching, which is used 
in teacher evaluation. Danielson (1996) suggested using the framework for supervision 
and evaluation. Thus, the framework and the accompanying principles identified by 
Danielson should be included in the teachers' and supervisors' workshops on teacher 
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evaluation so that the framework and the principles can be reviewed and discussed. 
Teachers and supervisors could review the framework to determine if some aspects could 
be adapted to the DLIFLC teacher-evaluation program. The supervisors' workshop 
should also include information regarding the supervisors' role in helping establish a 
culture of inquiry to promote professional development (Danielson, 2008). 

Lastly, a set of guidelines or a policy on teacher evaluation at DLIFLC should be 
developed. The guidelines should be prepared following the development and 
implementation of the workshops on teacher evaluation. The lessons learned from these 
valuable workshops will be of benefit to the authors of the DLIFLC teacher-evaluation 
guidelines. Input from DLIFLC teachers and supervisors regarding their perspectives on 
teacher-evaluation strategies and processes, as well as a review of the findings of this 
research study, should be incorporated into the guidelines. 

Recommendations for Future Research 

This study produced findings that included the perspectives of teachers and 
supervisors regarding the teacher-evaluation strategies and processes they use and 
experience. The findings led to implications: the following recommendations are for 
future study on teacher evaluation and its relationship to teachers' professional 
development. 

This mixed-methods study incorporated both quantitative and qualitative data. 
The written comments provided by participants added insight into teachers' and 
supervisors' views regarding the effectiveness of teacher-evaluation strategies. It will be 
beneficial, however, to conduct further qualitative studies, such as case studies that would 
include interviews and observations at DLIFLC and other training institutions that use the 
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Army's TAPES. Qualitative research approaches would generate additional insights into 
teachers' and supervisors' viewpoints on teacher evaluation. 

Another recommendation would be to replicate the study with a larger sample to 
provide generalizability. The TES for Teachers and the TES for Supervisors could be 
used and administered to DLIFLC undergraduate foreign-language teachers and 
supervisors or to a similar population. Additionally, both versions of the TES could be 
modified to include additional or alternate questions about the supervisor's role in teacher 
evaluation in order to gain additional insight into the supervisors' roles as 
transformational leaders. 

Additional recommendations for future research include: 

• Conducting studies on the various teacher-evaluation strategies and the 
contribution each makes to the promotion of teachers' professional 
development. 

• Examining the role of transformational leadership in teacher evaluation by 
investigating, in particular, the supervisors' role in providing effective 
feedback that promotes teachers' professional development. 

• Examining the role that gender plays when comparing the responses of male 
and female teachers regarding teacher-evaluation strategies, evaluation 
processes, and objectives of the teacher-evaluation system. 

• Examining the role that culture plays when comparing the responses of 
foreign-language teachers regarding teacher-evaluation strategies, the 
evaluation processes, and the objectives of the teacher-evaluation system. 
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• Determining the preferences of teachers concerning the teacher-evaluation 
strategies they believe are most effective in promoting their professional 
development. 

• Exploring the roles that supervisors play in coaching or mentoring teachers 
regarding their teaching performance and the impact supervisors have on the 
teachers' professional growth and development. 

• Conducting a longitudinal study to examine the impact of specific teacher- 
evaluation strategies, the feedback process used by supervisors, and the 
contribution of the strategies and processes to teachers' professional 
development. 

Concluding Thoughts 
Personnel evaluation has been described as being a difficult task that is often 
performed poorly and rarely performed well (JCSEE, 2009). Teachers and supervisors 
have agreed that teacher evaluation is an important part of the educational process; 
however, it is in the implementation of teacher-evaluation programs that teachers and 
supervisors have expressed concerns (Beerens, 2000; Danielson & McGreal, 2000). 
Teacher evaluation is a complex process with many facets including the various teacher- 
evaluation strategies used to collect evidence of teaching. Teacher evaluation also 
involves various processes that include the role of the supervisor and the performance 
standards used to evaluate teachers. Each of the factors or components that are a part of 
teacher evaluation play a vital role in ensuring that teacher evaluation is effective, that is, 
that it contributes to the professional development of the teacher. 
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A key aspect of teacher evaluation is the role of the supervisor. A supervisor who 
creates an environment that is one of trust and support contributes to the teacher being 
open to the supervisor's feedback (Acheson & Gall, 1997; Beerens, 2000; Holland, 2004; 
Stronge et al., 2008). Additionally, a supervisor who is knowledgeable and skilled in the 
various aspects of teacher evaluation will gain the confidence and respect of the teacher 
being evaluated (Davis et al., 2002). 

Involving teachers more in teacher evaluation will lead to more effective teacher 

evaluation. Emphasizing the connection between teacher evaluation and professional 

growth will enhance job performance and teacher satisfaction. 

Evaluation systems that promote professional growth through such vehicles as 
self-assessment, peer review, coaching, mentoring, and individual growth plans 
should lead to higher job performance levels among staff. Empowering evaluatees 
to play a key role in their own growth helps build trust and respect between 
evaluators and evaluatees and promote staff morale. (JCSEE, 2009, p. 94) 

Thus, an effective teacher-evaluation program is one that is linked to professional 
development. That is, an effective teacher-evaluation program is one that guides 
supervisors to become coaches and mentors who provide teachers opportunities to reflect 
on their teaching. 

Reflections of the Researcher 
In order to improve student achievement, teachers must be given opportunities to 
grow and develop as professional educators. Effective teacher-evaluation programs 
provide avenues for improvement of teaching performance. As previously reported, 
teacher evaluation serves two main purposes: (a) for accountability and (b) for 
improvement of teaching performance. Whereas the accountability focus has been 
emphasized over the years in education, the teaching-improvement aspect has often been 
neglected. Supervisors play a key role in ensuring that evaluation programs are effective. 
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The feedback processes used by supervisors and the quality of relationships they have 
built factor into an evaluation program being effective. Teachers and supervisors must 
work together during the formative and summative phases of the evaluation system. 

Supervisors and teachers engage in the process of formative and summative 
evaluations. Throughout the year, supervisors provide formative evaluations and through 
the process of giving constructive feedback they serve as coaches and mentors. 
Supervisors conduct many of the teacher-evaluation strategies described in this study, 
such as classroom observations, and collect data which are then shared with teachers. 
Thus, supervisors are able to base their formative evaluations on multiple sources of 
feedback, which is an important recommendation found in much of the scholarly 
literature on teacher evaluation (Beerens, 2000; Finch et al, 1997; Peterson, 2004; Seldin, 
1999; Stronge et al., 2008). 

In formative processes, supervisors discuss with foreign-language teachers the 
feedback they have obtained from the various teacher-evaluation strategies implemented. 
Classes are observed and objective observation data are reviewed and discussed. Students 
provide feedback regarding their teachers' quality of instruction; and teachers and 
supervisors read and analyze the data. Teachers participate in reflective practices and 
receive feedback to help them grow and develop. Some teachers participate in action 
research, which allows them to pose questions regarding their classroom practices and 
implement interventions to allow them to try new approaches to better help their students 
achieve their learning goals. The formative process of evaluation provides opportunities 
for teachers and supervisors to collaborate regarding the important objective of achieving 
the learning objectives. 
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Supervisors who strive to provide opportunities to engage in productive dialogue 
with teachers about their teaching performance implement behaviors reflective of 
transformational leaders. By providing constructive feedback and attending to the 
professional goals and interests of each teacher, the supervisor becomes an individually 
considerate leader; thereby attending to "each follower's needs for achievement and 
growth" (Bass, 2008, p. 622). Further, those supervisors who develop trusting 
relationships with teachers invest in teachers' developmental process, which in turn 
contributes to increased motivation on the part of the teachers. Teachers who are given 
the benefit of receiving constructive feedback improve their teaching performance and 
consequently contribute to improved student performance. In the formative aspect of 
evaluation, the supervisor is a trusted mentor who guides, offers suggestions, and listens 
to the teachers' concerns. 

The previous description of formative evaluation describes the ideal situation. 
Unfortunately, this ideal description of formative evaluation is not always achieved. 
Supervisors who evaluate teaching faculty are busy with many important duties and short 
cuts are often taken in the formative process. On occasion, supervisors neglect to provide 
the constructive feedback that is so critical to the teacher's development. 

With respect to summative evaluation, the researcher has observed over the span 
of years the tension that foreign-language teachers and their supervisors oftentimes 
experience when summative evaluations are prepared and given to the teacher. In the 
summative process of evaluations the supervisor becomes the judge, the evaluator, the 
one who determines the level of success the teacher will achieve in the rating period. 
Once the formal, summative evaluation is signed and the document becomes part of the 
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teachers' official record, the collaborative dialogue with a reflection on practice 
oftentimes diminishes or ceases completely. The focus on growth and development gets 
overshadowed by the summative-evaluation rating. In an effort to bring to light the 
missed opportunities to use the summative evaluation as a tool for growth and 
development, the researcher examined the perspectives of teachers and supervisors 
regarding teacher-evaluation strategies and processes. 

DLIFLC supervisors and teachers invest time in preparing summative evaluations. 
Teachers prepare their input, which is a type of self-evaluation. Teachers address each 
performance standard describing their contributions. They cite student comments, 
provide student results, and include references to their professional-development 
accomplishments. Supervisors also gather data to prepare the summative evaluation. 
They review the performance standards and compare the teachers' accomplishments to 
each standard. Once supervisors have examined the relevant data collected from the 
teacher-evaluation strategies such as classroom observations and SET, they prepare the 
narrative describing the teachers' contributions related to each standard. Finally, an 
overall assessment is assigned. Ideally, the summative evaluation should serve as a 
catalyst to initiate dialogue that leads to growth and development. In this way, the 
feedback from the summative evaluation leads to discussions of a formative nature. A 
cycle of formative evaluation to summative evaluation and back to formative evaluation 
should take place. 

The teacher-evaluation strategies and processes examined in this study should 
serve as opportunities for discussions about teaching and learning. The study's results 
have shown that many teacher-evaluation strategies do inform teachers' professional 
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development. The study's results further showed that the dialogues between supervisors 
and teachers are often productive and include objective suggestions for teachers' 
improvement. The results also indicated that the dialogues between supervisors and 
teachers regarding teaching effectiveness are, on occasion, rather one-sided and not 
reflective of two-way communication. 

Teachers and supervisors must consider the multiple sources of data used in 
teacher evaluation and engage in collaborative conversations that focus on reflection of 
teaching practice so that DLIFLC and the UEL will meet the developmental needs of its 
faculty. It is the view of this researcher that the time and effort invested by teachers and 
supervisors in teacher evaluation should lead to more fruitful and productive 
conversations about effective teaching and most importantly to improved teaching 
performance. 
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APPENDICES 



APPENDIX A 
DLIFLC PERMISSION TO CONDUCT STUDY 



DEPARTMENT OF THE ARMY 
DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER 
AND PRESIDIO OF MONTEREY 
MONTEREY CA 93944-3236 



10 September 20 10 



Institutional Review Board 

U.S. Army Assurance: DOD A20209 



Institutional Review Board for the Protection of Human Subjects 

University of San Francisco (USF) 

2130 Fulton Street 

San Francisco, C A 94117 



Dear Members of the USF IRB: 

On behalf of the U.S. Army Defense Language Institute Foreign Language Center (DLIFLC), I 
am writing to formally indicate our awareness of a research project proposed by Ms. Deanna 
Tovar, a graduate student at USF. 

This research project, tentatively entitled Teacher Evaluation: Foreign-language Teacher 's and 
Administrators ' Perspectives on Effective Evaluation Practices, has been reviewed by Dr. 
Donald Fischer (DLIFLC Provost), and he has approved the use of DoD personnel (military 
and civilian) as participants in this research project. 

I have been informed that the USF LRB will conduct the review and maintain institutional 
oversight of this project. Once the USF IRB has completed its review of the project, I ask that 
a copy of the outcome of that review (and approval number) be send to me so we may maintain 
a folder on this project in our file of current research projects. 

If you have any questions or concerns, please feel free to contact me. 




IRB Chair 

(831)393-1518 

jeff.crowson@us.army.mil 
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APPENDIX B 
USF HUMAN SUBJECTS APPROVAL 



IRB Application #10-115 - Approved 
USF IRBPHS - Nov 22 



November 22, 2010 
Dear Ms. Tovar: 

The Institutional Review Board for the Protection of Human Subjects 
(IRBPHS) at the University of San Francisco (USF) has reviewed your 
request for human subjects approval regarding your study. 

Your application has been approved by the committee (IRBPHS #10-115) . 
Please note the following: 

1. Approval expires twelve (12) months from the dated noted above. At 
that time, if you are still in collecting data from human subjects, you 
must file a renewal application. 

2. Any modifications to the research protocol or changes in 
instrumentation ( including wording of items) must be communicated to the 
IRBPHS. Re-submission of an application may be required at that time. 

3. Any adverse reactions or complications on the part of participants 
must be reported (in writing) to the IRBPHS within ten (10) working 
days . 

If you have any questions, please contact the IRBPHS at (415) 422-6091. 

On behalf of the IRBPHS committee, I wish you much success in your 
research . 

Sincerely, 

Terence Patterson, EdD, ABPP 

Chair, Institutional Review Board for the Protection of Human Subjects 



IRBPHS - University of San Francisco 

Counseling Psychology Department 

Education Building - Room 017 

2130 Fulton Street 

San Francisco, CA 94117-1080 

(415) 422-6091 (Message) 

(415) 422-5528 (Fax) 

irbphs@usf ca . edu 



http : / /www. usf ca . edu/soe/ students/ irbphs/ 



APPENDIX C 



PERMISSION TO ADAPT SURVEY 
In a message dated 11/10/2009 8:23:50 P. M. Pacific Standard Time, 



Hello Deanna- 
You 

have the right person and you are welcome to use whatever 
you want. You don't have to worry about giving 
me credit — that is not 
something that concerns me. 

Good luck and let me know if there is anything I can do for 
you. 

Heather McDougall 
On Tue, 11/10/09, \ 

i □ 

From: 



Subject: I am a USF doctoral student 



To: | 

Date: Tuesday, November 10, 2009, 9:14 PM 
Dear Heather, 

My name is Deanna Tovar and I am a doctoral student at USF and I have 

selected teacher evaluation as my topic for my dissertation. I am not sure if 

you are the Heather Mc Dougall who attended USF and did her dissertation in 

Dr. Mitchell is my advisor and she was also your advisor. 

If you are this Heather Mc Dougall, I am very 

interested in getting your permission to use your survey in my 

study. 

Please let me know if you are willing to allow me to 

use your survey and of course the proper citations and credits would be given. 

I may need to alter the 

questionnaire somewhat for my institution. 

I would greatly appreciate hearing back from you as I am very interested in 
your survey. It would fit perfectly into my study. 

Thank you in advance for your consideration. 

Sincerely, 

Deanna 
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APPENDIX D 

TEACHER EVALUATION SURVEY FOR TEACHERS 

The following anonymous survey measures your perceptions of the teacher evaluation 
system in your school. Please respond to each question as thoughtfully and completely as 
you can. Your responses will be strictly confidential. Please return the questionnaire to 
the survey administrator. Thank you in advance for your thoughtful responses. 

Part I A — Evaluation Strategies Used in the Teacher Evaluation Process 

1. Place an "X" next to each of the teacher evaluation strategies listed below that are 
used in your evaluations: 

A. Classroom observation by your supervisor for the purpose of evaluation 

B. Pre-observation discussion with your supervisor 

C. Post-observation conference with your supervisor 

D. Written observation report completed by your supervisor 

E. Student evaluations (such as Interim or End of Course Questionnaires, I/ESQs) 

F. Student feedback from sensing sessions 

G. Performance reviews (feedback & discussion with your supervisor) 

H. Students' performance on standardized tests (such as DLPT scores, discussed 

with your supervisor as part of the formal evaluation process) 

I. Professional portfolios (may contain artifacts such as lesson plans, teaching 

materials, evidence of student learning, and/or reflections on a teaching) 

J. Action Research (investigation of a teaching question/problem which includes 

the collection and analysis of data and a written report of your project) 

K. Self-evaluation (self-assessment involving reflection and judgment for the 

purpose of self -improvement as a teacher, e.g., may be in the form of a written report) 

Other (please explain) 
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Part I B — Evaluation and Professional Development 

2. For each of the evaluation strategies listed below, please indicate the extent to which 
you agree that the strategy contributes positively to your professional development 
using the following scale: 

SA: Strongly agree A: Agree U: Undecided D: Disagree SD: Strongly disagree 



In the space below each strategy, please explain what makes a given strategy 
effective or ineffective. 





SA 


A 


U 


D 


SD 


A. Classroom observation for the purpose of evaluation 













Comments: 



B. Pre-observation discussion with my supervisor 
Comments: 



C. Post-observation conference with my supervisor 
Comments: 



D. Written observation report completed by my supervisor 
Comments: 



E. Student evaluation (such as Interim or End of Course 
Questionnaires, I/ESQs) 
Comments: 



F. Student feedback from sensing sessions 
Comments: 



G. Performance reviews (feedback and discussion with my 
supervisor) 
Comments: 



225 



Part I B (Continued) — Evaluation and Professional Development 

2. For each of the evaluation strategies or sources of information listed below, please 
indicate the extent to which you agree that the strategy contributes positively to your 
professional development, using the following scale: 

SA: Strongly agree A: Agree U: Undecided D: Disagree SD: Strongly disagree 



In the space below each strategy, please explain what makes a given strategy 
effective or ineffective. 





SA 


A 


U 


D 


SD 


H. Students' performance on standardized tests (such as 
DLPT scores, discussed with my supervisor as part of the 
formal evaluation process) 













Comments: 



I. Professional portfolios (may contain artifacts such as lesson 
plans, teaching materials, evidence of student learning, 

and/or reflections on teaching) 

Comments: 



J. Action research (investigation of a teaching 
question/problem which includes the collection and analysis 

of data and a written report of your project) 

Comments: 



K. Self-evaluation (self-assessment involving reflection and 
Judgment for the purpose of self-improvement as a teacher, 
e.g., may be in the form of a written report) 
Comments: 



Please write any additional evaluation strategies or sources of information that you 
feel would contribute positively to your professional development and that you 
therefore recommend using: 
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Part II — Perceived Effectiveness of the Teacher Evaluation Process 

Using the Strongly agree . . . Strongly disagree scale provided, please indicate the extent 
to which you agree with the following statements. 

3. My supervisor is adequately trained to evaluate my teaching performance. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

4. My supervisor spends sufficient time in my classroom to evaluate my performance. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

5. 1 do not change my teaching style when my supervisor is observing my classroom. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

6. My evaluation is based on clearly defined performance standards. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

7. My evaluation is based on performance standards that promote better teaching. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

8. 1 receive timely feedback from my supervisor. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

9. My supervisor is accurate in his/her assessment of my teaching performance. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

10. My supervisor assesses all teachers fairly. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

11. My supervisor would listen to any changes I might suggest for the evaluation process. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

12. My evaluation process provides an opportunity to have a productive dialogue with my 
supervisor about my strengths and needed improvements as a teacher. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

13. My supervisor uses the evaluation process as a way to help me improve my teaching 
ability. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

14. The teacher evaluation process has a positive influence on my future teaching 
methods. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 
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15. During the evaluation process, I set professional development goals for the next 
teaching year with my supervisor. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

16. My supervisor and I discuss workshops/seminars/courses for me to attend to improve 
my teaching. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

17. My evaluation increases my confidence in my teaching ability. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

18. The evaluation process is more meaningful for non-tenured faculty. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

19. My school's evaluation system could be improved. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

For question 19, if you selected Strongly agree/or Agree, please explain how the 
system could be improved: 
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Part III — Perceived Objective of the Teacher Evaluation Process 

20. Improving teacher performance is an important objective of the evaluation process. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

21. Assessing quality of instruction is an important objective of the evaluation process. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

22. PLEASE CHECK ONLY ONE . The main purpose of my school's evaluation 
process is to: 

□ Assess my teaching performance 

□ Establish goals for the next school year 

□ Fulfill an administrative requirement 

□ Improve my teaching performance 

□ Promote self-reflection on my professional abilities 

□ Determine future monetary compensation (e.g. merit 
pay) 

□ Other (Please explain) 



23. PLEASE CHECK ONLY ONE . My school's evaluation process would be more 
useful to me if the main purpose were to : 

□ Assess my teaching performance 

□ Establish goals for the next school year 

□ Fulfill an administrative requirement 

□ Improve my teaching performance 

□ Promote self-reflection on my professional abilities 

□ Determine future monetary compensation (e.g., 
merit pay) 

□ Other (Please explain.) 
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Part IV — Background and Demographics 

This information will be used to give an accurate description of the population being 
surveyed. 

24. 1 am 



25. 1 am 



□ Male 

□ Female 



26. 1 am 



□ 21-30 years old 

□ 31-40 years old 

□ 41-50 years old 

□ 51-60 years old 

□ Over 60 years old 



□ Tenured 

□ Non-tenured 



27. Please check your academic rank 

□ Professor 

□ Associate Professor 

□ Assistant Professor 

□ Senior Instructor 

□ Instructor 

□ Assistant Instructor 
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28. Please check the highest degree you have earned 

□ Bachelor's Degree 

□ Master's Degree 

□ Ed.D. or Ph.D. 

29. Total foreign-language teaching experience 

□ 0-5 Years 

□ 6-10 Years 

□ 11-15 Years 

□ 16-20 Years 

□ 21-25 Years 

□ 26-30 years 

□ More than 30 Years 

30. Number of years of teaching experience at DLIFLC is 

□ 0-5 Years 

□ 6-10 Years 

□ 11-15 Years 

□ 16-20 Years 

□ 21-25 Years 

□ 26-30 years 

□ More than 30 Years 



31. My supervisor is 



32. My supervisor is 



□ My Department Chair 

□ Other (Please Specify.) 



□ Female 

□ Male 



Thank you for your participation! I would greatly appreciate any comments you 
may have on the survey as a whole and/or specific survey items in order to make the 



survey clearer to teachers who may complete it in the future. 
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APPENDIX E 

TEACHER EVALUATION SURVEY FOR SUPERVISORS 

The following anonymous survey measures your perceptions of the evaluation process, as 
well as your practices as an evaluator. Please respond to each question as it applies to 
your experience as a supervisor. Your responses will be strictly confidential. Please 
return the questionnaire to the survey administrator. Thank you in advance for your 
thoughtful responses. 

Part I A — Evaluation Strategies Used in the Teacher Evaluation Process 

1. Place an "X" next to the teacher evaluation strategies listed below that you use in 
your evaluation of teachers: 

A. Classroom observation for the purpose of evaluation 

B. Pre-observation discussion with the teacher 



C. Post-observation conference with the teacher 



D. Written observation report that I complete 

E. Student evaluation (such as Interim or End of Course Questionnaires, I/ESQs) 

F. Student feedback from sensing sessions 

G. Performance reviews (feedback & discussion with the teacher) 

H. Students' performance on standardized tests (such as DLPT scores, discussed 

with the teacher as part of the formal evaluation process) 

I. Professional portfolios (may contain artifacts such as lesson plans, teaching 

materials, evidence of student learning, and/or reflections on teaching) 

J. Action research (investigation of a teaching question/problem which includes 

the collection and analysis of data and a written report of the teachers' project) 

K. Teacher self-evaluation (self-assessment involving reflection and judgment 

for the purpose of self-improvement as a teacher, e.g., may be in the form of a written 
report) 

Other (Please explain.) 
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Part I B — Evaluation and Professional Development 

2. For each of the evaluation strategies listed below, please indicate the extent to which you agree 
that the strategy contributes positively to your teachers' professional development, using the 
following scale: 

SA: Strongly agree A: Agree U: Undecided D: Disagree SD: Strongly disagree 



In the space below each strategy, please explain what makes a given strategy effective or 
ineffective. 



SA 



U 



D 



SD 



A. Classroom observation for the purpose of evaluation 



Comments: 



B. Pre-observation discussion with the teacher 
Comments: 



C. Post-observation conference with the teacher 
Comments: 



D. Written observation report that I complete 
Comments: 



E. Student evaluation (such as Interim or End of 

Course Questionnaires, I/ESQs) 

Comments: 



F. Student feedback from sensing sessions 
Comments: 



G. Performance reviews (feedback and discussion with 
the teacher) 
Comments: 
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Part I B (Continued — Evaluation and Professional Development 

2. For each of the evaluation strategies or sources of information listed below, please indicate the 
extent to which you agree that the strategy contributes positively to your teachers' professional 
development, using the following scale: 

SA: Strongly agree A: Agree U: Undecided D: Disagree SD: Strongly disagree 



In the space below each strategy, please explain what makes a given strategy effective or 
ineffective. 





SA 


A 


U 


D 


SD 


H. Students' performance on standardized tests (such as 
DLPT scores, discussed with me as part of the 
formal evaluation process) 













Comments: 



I. Professional portfolios (Teachers' portfolios may contain 
artifacts such as lesson plans, teaching materials, evidence 

of student learning, and/or reflections on teaching) 

Comments: 



J. Action research (investigation of a teaching 
question/problem which includes the collection and analysis of 
data and a written report of your project) 



Comments: 



-evaluation (self-assessment involving reflection and judgment 
for the purpose of self-improvement as a teacher, e.g. may be in 
the form of a written report) 
Comments: 



Please write any additional evaluation strategies or sources of information that you 
feel would contribute positively to your teachers' professional development and that 
you therefore recommend using: 
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Part II — Perceived Effectiveness of the Teacher Evaluation Process 

Using the Strongly agree . . . Strongly disagree scale provided, please indicate the extent 
to which you agree with the following statements: 

3. 1 am adequately trained to evaluate my teachers' performance. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

4. 1 spend sufficient time conducting classroom observations to evaluate my teachers' 
performance. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

5. My teachers do not change their teaching style to meet my expectations when I observe 
them. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

6. My evaluations of my teachers are based on clearly defined performance standards. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

7. My evaluations of my teachers are based on performance standards that promote better 
teaching. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 
8. 1 provide timely feedback to my teachers. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

9. 1 am accurate in my assessment of my teachers' performance. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

10. 1 assess all teachers fairly. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

11. 1 would listen to any changes my teachers might suggest for the evaluation process. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

12. My evaluation process provides an opportunity for me to have a productive dialogue 
with my teachers about their strengths and needed improvements. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

13. 1 use the evaluation process as a way to help my teachers improve their teaching 
ability. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 
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14. The teacher evaluation process has a positive influence on my teachers' future 
teaching methods. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

15. During the evaluation process, I ask my teachers to set professional development 
goals for the next teaching year. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

16. My teachers and I discuss workshops/seminars/courses they might attend to improve 
their teaching. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

17. My evaluation increases my teachers' confidence in their teaching ability. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

18. The evaluation process is more meaningful for non-tenured faculty. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

19. My school's evaluation system could be improved. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

For question 19, if you selected Strongly Agree/ Agree, please explain how the system 
could be improved: 
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Part III — Perceived Objective of the Teacher Evaluation Process 

20. Improving teacher performance is an important objective of the evaluation process. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

21. Assessing quality of instruction is an important objective of the evaluation process. 

□ Strongly agree □ Agree □ Undecided □ Disagree □ Strongly disagree 

22. PLEASE CHECK ONLY ONE. The main purpose of my school's evaluation process 
is to: 

□ Assess teaching performance 

□ Establish goals for the next school year 

□ Fulfill an administrative requirement 

□ Improve teaching performance 

□ Promote self-reflection on my teachers' professional 
abilities 

□ Determine future monetary compensation (e.g., 
merit pay) 

□ Other (Please explain) 



23. PLEASE CHECK ONLY ONE . My school's evaluation process would be more 
useful if the main purpose were to: 

□ Assess teaching performance 

□ Establish goals for the next school year 

□ Fulfill an administrative requirement 

□ Improve teaching performance 

□ Promote self-reflection on my teachers' professional 
abilities 

□ Determine future monetary compensation (e.g., 
merit pay) 

□ Other (Please explain.) 
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Part IV — Background and Demographics 

This information will be used to give an accurate description of the population being 

surveyed. 

24. 1 am 



25. 1 am 



□ Male 

□ Female 



□ 21-30 years old 

□ 31-40 years old 

□ 41-50 years old 

□ 51-60 years old 

□ Over 60 years old 

26. 1 am 



□ Tenured 

□ Non-tenured 

27. Please check your academic rank 



□ Professor 

□ Associate Professor 

□ Assistant Professor 

□ Senior Instructor 

□ Instructor 

□ Assistant Instructor 



28. Please check the highest degree you have earned 

□ Bachelor's Degree 

□ Master's Degree 

□ Ed.D. or Ph.D. 



29. Total foreign-language teaching experience 

□ 0-5 Years 

□ 6-10 Years 

□ 11-15 Years 

□ 16-20 Years 
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□ 21-25 Years 

□ 26-30 years 

□ More than 30 Years 

30. Number of years of teaching experience at DLIFLC is 

□ 0-5 Years 

□ 6-10 Years 

□ 11-15 Years 

□ 16-20 Years 

□ 21-25 Years 

□ 26-30 years 

□ More than 30 Years 

31. Number of years of supervisory experience at DLIFLC is 

□ 0-5 Years 

□ 6-10 Years 

□ 11-15 Years 

□ 16-20 Years 

□ 21-25 Years 
LI 26-30 years 

□ More than 30 Years 

Thank you for your participation! 

I would greatly appreciate any comments you may have on the survey as a whole 
and/or specific survey items in order to make the survey clearer to supervisors who 



may complete it in the future. Thank you for your comments: 
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APPENDIX F 

VALIDITY PANEL DEMOGRAPHICS FOR TES FOR TEACHERS 



Validity Panel Demographics: Teacher Evaluation Survey for Teachers 



Gender 


Education 


Experience 
conducting 
evaluations 


Experience as 
teacher 
developer 


Current and previous positions 
neiu 


F 


M.A. 


Yes 


Yes 


Academic Specialist/Curriculum 
Coordinator, Foreign Language 
1 eacner, 1 earn Leader, 1 est 
Writer, Department Chair 


M 


Ill, T 

rn.D. 


Yes 


Yes 


Academic Specialist, Teacher, 
Team Leader, Department Chair, 
Course Developer 


F 


M.A. 


Yes 


Yes 


Academic Specialist, Language 
Teacher, Department Chair, Test 
Writer, Faculty Development 
Specialist 


F 


M.A. 


Yes 


Yes 


Academic Specialist, Technology 
Trainer, Team Leader, Foreign 
Language Teacher 


M 


Ph.D. 


No 


Yes 


Senior Research Scientist, Foreign 
Language Teacher 
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APPENDIX G 

VALIDITY PANEL DEMOGRAPHICS FOR TES FOR SUPERVISORS 



Validity Panel Demographics: Teacher Evaluation Survey for Supervisors 



Gender 


Education 


Experience 
conducting 
evaluations 


Experience as 
teacher developer 


Current and previous positions 
held 


F 


Ph.D. 


Yes 


Yes 


Senior Research Scientist, 
University Professor 


F 


Ed.D. 


Yes 


Yes 


Dean of Language School, 
Foreign Language Teacher, 
Department Chair, Curriculum 
Specialist 


M 


Ph.D. 


Yes 


No 


Dean of Language School, 
ESL/TESOL Professor, 
Division Chair 


F 


Ph.D. 


Yes 


No 


Dean of Language School, 
Foreign Language Teacher, Test 
Writer, Department Chair, 
Team Leader 
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APPENDIX H 
INFORMED CONSENT FORM 



INFORMED CONSENT FORM 

UNIVERSITY OF SAN FRANCISCO 
CONSENT TO BE A RESEARCH SUBJECT 
Purpose and Background 

Ms. Deanna Tovar, a graduate student in the School of Education at the University of 
San Francisco is doing a study on foreign-language teachers' and administrators' 
perspectives on effective teacher evaluation processes. 

I am being asked to participate because I am a foreign-language teacher or administrator 
at the Defense Language Institute Foreign Language Center (DLIFLC). 

Procedures 

If I agree to be a participant in this study, the following will happen: 

1. 1 will complete a short questionnaire giving basic information about me, 
including age, gender, academic rank, employment status (tenured or non- 
tenured) and experience teaching foreign languages. 
2. 1 will complete a survey about teacher evaluation processes at the DLIFLC. 

Risks and/or Discomforts 

1. It is possible that some of the questions on the teacher evaluation survey may 
make me feel uncomfortable, but I am free to decline to answer any questions 
I do not wish to answer or to stop participation at any time. 

2. Participation in research may mean a loss of confidentiality. Study records 
will be kept as confidential as is possible. No individual identities will be used 
in any reports or publications resulting from the study. Study information will 
be coded and kept in locked files at all times. Only study personnel will have 
access to the files. 

Benefits 

There will be no direct benefit to me from participating in this study. The anticipated 
benefit of this study is a better understanding of teacher evaluation processes. 

Costs/Financial Considerations 

There will be no financial costs to me as a result of taking part in this study. 
Questions 

If I have any questions or comments about part icipation in thi s study, I will first talk with 



the researcher, Ms. Deanna Tovar by phone at < or by email at 



If for some reason I do not wish to do this, I may contact 
the IRBPHS, which is concerned with protection of volunteers in research projects. I 
may reach the IRBPHS office by calling (415) 422-6091 and leaving a voicemail 
message, by e-mailing IRBPHS@usfca.edu, or by writing to the IRBPHS, Department of 
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Psychology, University of San Francisco, 2130 Fulton Street, San Francisco, CA 941 17- 
1080. 

Consent 

I have been given a copy of the "Research Subject's Bill of Rights" and I have been 
given a copy of this consent form to keep. 

PARTICIPATION IN RESEARCH IS VOLUNTARY. I am free to decline to be in this 
study, or to withdraw from it at any point. My decision as to whether or not to participate 
in this study will have no influence on my present or future status at the DLIFLC. 
My signature below indicates that I agree to participate in this study. 



Subject's Signature Date of Signature 



Signature of Person Obtaining Consent 



Date of Signature 
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APPENDIX I 

RESEARCH SUBJECTS' BILL OF RIGHTS 

UNIVERSITY OF SAN FRANCISCO CONSENT TO BE A RESEARCH 

SUBJECT 

Research Subjects' Bill of rights 

The rights listed below are the rights of individuals who are asked to participate in a 
research study. 

As a research subject, I have the following rights: 

1 . To be told the extent to which confidentiality of records identifying the 
subject will be maintained and of the possibility that specified individuals, 
internal and external regulatory agencies, or study sponsors may inspect 
information in the medical record specifically related to participation in the 
clinical trial. 

2. To be told of any benefits that may reasonably be expected from the research. 

3. To be told of any reasonable foreseeable discomforts or risks. 

4. To be told of appropriate alternative procedures or courses of treatment that 
might be of benefit to the subject. 

5. To be told of the procedures to be followed during the course of participation, 
especially those that are experimental in nature. 

6. To be told that they may refuse to participate (participation is voluntary), and 
that declining to participate will not compromise access to services and will 
not result in penalty or loss of benefits to which the subject is otherwise 
entitled. 

7. To be told about compensation and medical treatment if research related 
injury occurs and where further information may be obtained when 
participating in research involving more than minimal risk. 

8. To be told whom to contact for answers to pertinent questions about the 
research, about the research subjects' rights and whom to contact in the event 
of a research-related injury to the subject. 

9. To be told of anticipated circumstances under which the investigator without 
regard to the subject's consent may terminate the subject's participation. 

10. To be told of any additional costs to the subject that may result from 
participation in the research. 

1 1. To be told of the consequences of a subjects' decision to withdraw from the 
research and procedures for the orderly termination of participation by the 
subject. 

12. To be told that significant new findings developed during the course of the 
research that may relate to the subject's willingness to continue participation 
will be provided to the subject. 

13. To be told the approximate number of subjects involved in the study. 

14. To be told what the study is trying to find out. 
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15. To be told what will happen to me and whether any of the procedures, drugs, 
or devices are different from what would be used in standard practice. 

16. To be told about the frequent and/or important risks, side effects, or 
discomforts of the things that will happen to me for research purposes. 

17. To be told if I can expect any benefit from participating, and if so, what the 
benefit might be. 

18. To be told of the other choices I have and how they may be better or worse 
than being in the study. 

19. To be allowed to ask any questions concerning the study both before agreeing 
to be involved and during the course of study. 

20. To be told what sort of medical or psychological treatment is available if any 
complications arise. 

21. To refuse to participate at all or to change my mind about participation after 
the study is started; if I were to make such as decision, it will not affect my 
right to receive the care or privileges I would receive if I were not in the study. 

22. To receive a copy of the signed and dated consent form; and 

23. To be free of pressure when considering whether I wish to agree to be in the 
study. 

If I have other questions, I should ask the researcher. In addition, I may contact the 
Institutional Review Board for the Protection of Human Subjects (IRBPHS), which is 
concerned with protection of volunteers in research projects. I may reach the IRBPHS by 
calling (415) 422-6091, by electronic mail at IRBPHS@usfca.edu, or by writing to USF 
IRBPHS, Department of Counseling Psychology, Education Building, 2130 Fulton Street, 
San Francisco, CA 94117-1080. 



